-- Logs begin at Tue 2022-08-02 08:28:55 UTC, end at Tue 2022-08-02 10:00:38 UTC. -- Aug 02 08:56:27.949714 dv-cinder-803823-15-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 02 08:56:27.962030 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 02 08:56:27.963957 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 2 08:56:27 2022] *** Aug 02 08:56:27.963957 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 02 08:56:27.963957 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 02 08:56:27.963957 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: nodename: dv-cinder-803823-15-iscsi Aug 02 08:56:27.963957 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: machine: x86_64 Aug 02 08:56:27.963957 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: clock source: unix Aug 02 08:56:27.963957 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: pcre jit disabled Aug 02 08:56:27.963957 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: detected number of CPU cores: 4 Aug 02 08:56:27.963957 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: current working directory: / Aug 02 08:56:27.963957 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: detected binary path: /usr/bin/uwsgi-core Aug 02 08:56:27.964485 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: your processes number limit is 47680 Aug 02 08:56:27.964485 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: your memory page size is 4096 bytes Aug 02 08:56:27.964485 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: detected max file descriptor number: 2048 Aug 02 08:56:27.964485 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: lock engine: pthread robust mutexes Aug 02 08:56:27.964485 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: thunder lock: enabled Aug 02 08:56:27.964485 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 02 08:56:27.964485 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 02 08:56:28.003929 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: Python main interpreter initialized at 0x55d5e457d9c0 Aug 02 08:56:28.003929 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: python threads support enabled Aug 02 08:56:28.003929 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: your server socket listen backlog is limited to 100 connections Aug 02 08:56:28.003929 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: your mercy for graceful operations on workers is 90 seconds Aug 02 08:56:28.004258 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: mapped 403077 bytes (393 KB) for 2 cores Aug 02 08:56:28.004336 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: *** Operational MODE: preforking *** Aug 02 08:56:28.004595 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: *** uWSGI is running in multiple interpreter mode *** Aug 02 08:56:28.004595 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: spawned uWSGI master process (pid: 80351) Aug 02 08:56:28.004773 dv-cinder-803823-15-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 02 08:56:28.005852 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: spawned uWSGI worker 1 (pid: 80361, cores: 1) Aug 02 08:56:28.005852 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: spawned uWSGI worker 2 (pid: 80362, cores: 1) Aug 02 08:56:28.005852 dv-cinder-803823-15-iscsi devstack@keystone.service[80351]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 02 08:56:33.592772 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 08:56:33.592772 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: from cryptography.utils import int_from_bytes Aug 02 08:56:33.593594 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 08:56:33.593594 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: from cryptography.utils import int_from_bytes Aug 02 08:56:33.594189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 08:56:33.594189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: from cryptography.utils import int_from_bytes Aug 02 08:56:33.594632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 08:56:33.594632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: from cryptography.utils import int_from_bytes Aug 02 08:56:33.643730 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 02 08:56:33.643918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 02 08:56:33.644111 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] command line args: [] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 02 08:56:33.644379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 02 08:56:33.644463 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 02 08:56:33.644542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ================================================================================ {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 02 08:56:33.644674 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 02 08:56:33.644799 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] admin_token = **** {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.644884 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] command line args: [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 02 08:56:33.645043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] config_dir = [] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.645181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 02 08:56:33.645257 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.645340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ================================================================================ {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 02 08:56:33.645421 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] config_source = [] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.645587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] admin_token = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.645665 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] debug = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.645783 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.646051 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] config_dir = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.646051 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.646171 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.646230 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] config_source = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.646375 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.646482 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] debug = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.646589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.646721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.646822 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.646922 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.647021 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.647134 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] list_limit = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.647234 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] log_config_append = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.647327 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.647419 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.647523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.647618 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] log_dir = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.647717 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.647818 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] log_file = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.647917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_config_append = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.648011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.648136 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.648218 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.648340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_dir = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.648340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_file = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.648524 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.648604 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.648680 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.648804 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.648887 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.648965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.649082 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.649218 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.649291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.649419 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.649503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.649628 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.649711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.649829 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.649906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.650061 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.650180 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.650294 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.650466 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.650541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.650659 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.650742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.650821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.650974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.651100 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.651224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.651473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.651556 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.100/identity {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.651673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] publish_errors = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.651752 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.651897 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.652128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.652238 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.100/identity {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.652322 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.652424 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] publish_errors = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.652542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.652661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.652787 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.652904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.653007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] standard_threads = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.653139 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.653270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.653375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.653529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.653614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.653692 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.653829 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] standard_threads = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.653904 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] use_journal = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.653973 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.654042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] use_json = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.654180 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] use_stderr = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.654265 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.654357 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] use_syslog = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.654434 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.654558 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] use_journal = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.654680 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.655287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] use_json = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.655469 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] use_stderr = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.655609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.655689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] use_syslog = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.655858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.655941 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:56:33.656124 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.656208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.656367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.656444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.656618 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.656696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.656859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.656946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.657132 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.657221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.657372 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.657450 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.657618 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.657702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.657858 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.657934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.658107 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.658186 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.658345 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.658433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.658593 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.658670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.658828 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.658909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.659076 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.659159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.659318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.659394 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.659551 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.659632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.659805 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.659962 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.660042 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.660112 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.660286 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.660362 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.660520 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.660604 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.660774 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.660851 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.661012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.661088 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.661303 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.661393 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.661554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.661638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.661786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.661862 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.662018 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.662107 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.662257 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.662375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.662483 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.662561 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.662735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.662814 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.662982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.663059 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.663211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.663288 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.663440 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.663518 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.663685 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.663762 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.663918 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.663994 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.664152 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.664229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.664397 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.664474 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.664625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.664701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.664861 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.664940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.665095 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.665196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.665348 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.665435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.665585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.665659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.665794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.665872 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.666022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.666099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.666253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.666331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.666488 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.666565 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.666715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.666790 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.666950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.667029 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.667180 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.667268 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.667416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.667494 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.667647 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.667751 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.667889 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.667992 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.668116 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.668218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.668450 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.668573 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.668651 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.668722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.668844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.668949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.669078 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.669178 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.669251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.669377 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.669493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.669568 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.669714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.669840 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.669951 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.670028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.670221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.670336 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.670458 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.670549 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.670768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.670958 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.671034 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.671101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.671180 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.671253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.671394 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.671500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.671629 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.671739 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.671895 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.671973 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.672132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.672217 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.672378 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.672462 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.672628 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.672708 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.672864 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.672941 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.673099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.673302 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.673426 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.673564 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.673686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.673810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.673935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.674073 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.674201 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.674336 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.674461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.674587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.674714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.674840 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.674966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.675100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.675233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.675361 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.675488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.675613 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.675732 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.675871 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.675999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.676124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.676252 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.676377 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.676503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.676642 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.676764 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.676890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.677016 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.677156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.677279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.677483 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.677624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.677751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.677883 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.678009 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.678135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.678269 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.678403 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.678536 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.678659 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.678781 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.678906 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.679033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.679158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.679291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.679428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.679562 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.679690 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.679816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.679943 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.680077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.680204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.680336 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.680463 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.680589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.680716 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.680850 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.680976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.681102 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.681252 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] auth.external = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.681378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.681517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.681651 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.681779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.681906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.external = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.682044 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.682199 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.682334 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] auth.password = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.682462 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.682598 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] auth.token = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.682726 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.682852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.682989 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.password = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.683124 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.683251 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.token = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.683386 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.683513 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.683636 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.683760 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.683888 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.684014 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.684152 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.684280 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.684406 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.684533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.684661 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] credential.caching = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.684798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.684933 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.685061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.685222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.685344 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.685480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.685607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.685743 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.685903 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.686170 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.686307 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.686435 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.686563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.686709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.686841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.686976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.687104 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.687231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.687357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.687492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.687625 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.687753 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.687880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.688012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.688140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.688275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.688403 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.688547 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.688671 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.688793 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.688920 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.689056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] federation.caching = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.689193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.689317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.689444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.689572 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.689697 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.689834 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.689961 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.690090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.690226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.690353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.690481 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.690615 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.690741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.690867 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.691001 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.691130 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.691263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.691387 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.691510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.691633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.691755 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.691878 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.692008 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.692141 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] identity.caching = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.692265 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.692387 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.692387 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.692617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.692815 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.692940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.693062 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.693236 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.693365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.693491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.693629 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.693711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.693849 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.693971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.694092 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.694218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.694348 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.694476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.694603 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.694728 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.694854 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.694981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.695115 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.695241 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.695368 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.695504 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.695631 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.695757 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.695891 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.696017 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.696143 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.696270 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.696397 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.696530 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.696665 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.696793 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.696920 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.697046 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.697200 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.697325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.697459 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.697588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.697714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.697835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.697967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.698093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.698227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.698354 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.698481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.698615 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.698739 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.698860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.699003 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.699121 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.699242 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.699367 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.699495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.699623 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.699757 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.699884 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.700010 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.700135 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.700258 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.700379 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.700509 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.700643 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.700767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.700889 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.701011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.701183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.701321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.701321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.702073 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.702197 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.702323 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.702450 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.702583 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.702712 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.702838 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.702975 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.703102 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.703230 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.703369 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.703497 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.703624 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.703769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.703891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.704017 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.704153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.704281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.704407 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.704533 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.704660 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.704799 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.704933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.705061 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.705201 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.705324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.705451 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.705578 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.user = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.705714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.705843 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.705969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.705969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.706215 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.706355 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.706483 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.706620 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.706747 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.706874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.707001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.707138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.707266 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.707394 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.707520 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.707647 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.707774 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.707908 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.708036 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.708036 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.708284 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.708413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.708538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.708673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.708789 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.708912 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.709034 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.709200 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.709323 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.709323 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.709575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.709702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.709828 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.709954 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.710091 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.710229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.710356 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.710484 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.710610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.710736 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.710862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.710996 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.711124 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.711250 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.711375 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.711502 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.711627 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.711771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.711898 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.712025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.712151 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.712277 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.712403 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.712403 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.712650 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.712782 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.712908 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.713030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.713171 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.713307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.713431 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.713568 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.713695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.713822 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.713948 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.714082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.714210 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.714336 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.714469 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.714597 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.714742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.714869 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.714992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.715114 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] resource.caching = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.715235 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.715371 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.715498 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.715634 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.715761 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.715888 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.716013 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.716139 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.716266 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.716401 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.716529 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.716656 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.716782 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.716908 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.717030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.717815 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.717942 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.718065 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.718242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.718328 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] role.caching = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.718451 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.718647 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] role.driver = sql {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.718777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.718900 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.719026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.719153 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.719279 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.719415 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.719555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] role.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.719682 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.719809 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] role.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.719936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.720063 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.720197 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.720331 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.720459 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.720586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.720713 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.720848 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.720975 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.720975 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.721228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.721361 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.721484 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.721619 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.721743 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.721865 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.721988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.722110 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.722233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.722233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.722471 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.722603 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.722725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.722847 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.722980 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.723111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.723235 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.723358 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.723481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.723481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.723713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.723843 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.723966 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.724089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.724211 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.724334 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.724457 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.724597 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.724722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.724845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.724968 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.725090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.725257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.725395 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.725523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.725650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.725784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.725911 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.726045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.726173 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.726308 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.726436 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.726562 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.726689 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.726825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.726952 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.727079 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.727205 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] token.caching = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.727331 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.727457 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.727457 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.727710 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.727832 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.727958 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.727958 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.728207 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.728341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.728469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.728595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.728722 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.728848 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.728971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.728971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.729228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.729228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.729459 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.729459 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.729699 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.729833 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.729957 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.730080 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.730202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.730324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.730447 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.730577 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.730701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.730824 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.730947 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.731070 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.731198 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.731329 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.731462 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.731825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] database.connection = **** {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.731963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.732095 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.732219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.732342 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.732465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.732600 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.732757 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.732884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.733067 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.733260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.733384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.733523 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.733661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.connection = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.733798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.733934 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.734063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.734192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.734319 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.734454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.734582 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.734708 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.734835 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.734961 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.735088 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.735224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.735358 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.735495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.735622 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.735748 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.735875 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.736009 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.736136 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.736264 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.736391 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.736518 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.736644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.736644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 02 08:56:33.736893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.736893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.736893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.736893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.736893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.736893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.736893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:56:33.736893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 02 08:56:33.759263 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.759488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.759709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.759839 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.760029 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.760256 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.768642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.769250 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.799196 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.799444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.799922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.800160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.852932 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.853376 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.904348 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.904649 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.905553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.905794 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.971426 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.971641 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.971856 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.972045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.972204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.972344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.972477 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.972612 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.972756 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.972756 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.973026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.973182 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.995936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.996109 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.996258 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.996400 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.996529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.996657 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.996784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.996922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.997059 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.997205 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.997334 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.997459 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.997584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:33.997711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:34.009158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:34.009388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:34.021982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:34.022209 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:34.022389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:34.022534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:34.022663 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:34.022791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:34.022917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:34.023047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:34.023186 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:34.023318 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:34.023449 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:34.023581 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:34.023724 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:34.023724 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:56:34.040561 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.040769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 08:56:34.041101 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.041259 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.041390 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 08:56:34.041537 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 08:56:34.041670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.041795 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.041918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.042051 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 08:56:34.042189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.042318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.042440 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.042561 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.042688 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.042936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.052013 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.052291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.052561 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.052686 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.052843 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.052992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.053266 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.053553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.059732 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.060449 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.064938 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.065252 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.065536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.065742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.065869 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.066031 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.066171 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.066320 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.066447 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.066580 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.066703 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.066835 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.066959 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.067083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.067207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.067330 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.067463 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.067596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.067720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.067917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.068194 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.068464 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.090873 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.091897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.092901 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.093206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.094044 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.094335 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.097779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.098062 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.098343 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.098978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.099264 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.099551 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.104982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.105254 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.106151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.106441 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.109983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.110276 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.110556 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.110846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.111129 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.111304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.111390 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.111608 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.111690 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.111899 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.111980 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.112188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.112469 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.112765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.113050 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.113357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.127667 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.127945 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.129827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.130112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.132098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.134048 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.135253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.135545 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.137227 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.137531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.139950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.142325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.143146 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.145575 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.146330 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.146627 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.146913 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.147217 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.147499 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.147780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.148747 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.149052 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.149369 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.149672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.149963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.150254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.164484 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.164766 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.165033 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.165320 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.166940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.167229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.167508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.167778 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.172915 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.174551 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.174823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.175328 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.176885 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.177192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.177653 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.177935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.178224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.178559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.178846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.180027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.180314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.180621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.180911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.181247 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.190708 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.190986 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.191262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.191540 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.191817 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.192136 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.192413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.193238 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.193523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.193807 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.194090 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.194372 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.194651 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.194929 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.208468 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.211101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.211439 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.214113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.214373 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.216800 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.217091 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.218526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.218808 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.219100 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.219586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.221163 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.221457 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.221741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.225534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.227126 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.228219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.229868 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.230053 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.230342 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.230643 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.230922 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.232821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.233178 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.233427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.233719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.239052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.239338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.239617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.241771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.242064 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.242349 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.246413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.246692 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.246971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.247246 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.247530 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.247808 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.248080 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.248935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.249525 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.249743 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:56:34.249825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.250026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.250105 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.250310 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.250398 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.250589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.250669 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.250855 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.250984 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.251151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.251427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.251714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.252099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.252384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.252673 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.252950 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.253243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.253775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:56:34.284768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d5e457d9c0 pid: 80361 (default app) Aug 02 08:56:34.287694 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d5e457d9c0 pid: 80362 (default app) Aug 02 08:56:34.288031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccceba6f-93a9-41e2-847b-db42e779146b None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:34.288320 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccceba6f-93a9-41e2-847b-db42e779146b None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:34.288597 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccceba6f-93a9-41e2-847b-db42e779146b None None] PATH_INFO: `/v3` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:34.292913 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1/1] 10.222.0.100 () {50 vars in 796 bytes} [Tue Aug 2 08:56:34 2022] GET /identity/v3/ => generated 256 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:56:41.899984 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5162701-efcd-4fae-b2ff-eb84fd975e6e None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:41.900547 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5162701-efcd-4fae-b2ff-eb84fd975e6e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:41.900547 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5162701-efcd-4fae-b2ff-eb84fd975e6e None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:41.905399 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1/2] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:56:41 2022] GET /identity => generated 271 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:56:41.910760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d6e32-b8e9-4099-9054-6afae79dc984 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:41.911171 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d6e32-b8e9-4099-9054-6afae79dc984 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:41.911306 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d6e32-b8e9-4099-9054-6afae79dc984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:41.980552 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: INFO dbcounter [None req-493d6e32-b8e9-4099-9054-6afae79dc984 None None] Registered counter for database keystone Aug 02 08:56:42.000303 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG oslo_db.sqlalchemy.engines [None req-493d6e32-b8e9-4099-9054-6afae79dc984 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80361) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 02 08:56:42.006704 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writer thread running {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 02 08:56:42.184176 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG passlib.handlers.bcrypt [None req-493d6e32-b8e9-4099-9054-6afae79dc984 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80361) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 02 08:56:42.184512 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG passlib.handlers.bcrypt [None req-493d6e32-b8e9-4099-9054-6afae79dc984 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80361) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 02 08:56:42.239528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-493d6e32-b8e9-4099-9054-6afae79dc984 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:56:42.241194 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-493d6e32-b8e9-4099-9054-6afae79dc984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:56:42.375288 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 2/3] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:56:41 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 02 08:56:42.501837 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5162701-efcd-4fae-b2ff-eb84fd975e6e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:42.572698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: INFO dbcounter [None req-c5162701-efcd-4fae-b2ff-eb84fd975e6e None None] Registered counter for database keystone Aug 02 08:56:42.594578 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG oslo_db.sqlalchemy.engines [None req-c5162701-efcd-4fae-b2ff-eb84fd975e6e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80362) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 02 08:56:42.600651 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writer thread running {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 02 08:56:42.734805 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5162701-efcd-4fae-b2ff-eb84fd975e6e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:42.739187 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a912d2e-5004-4cb9-88de-8dcb7bf24701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:42.739824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a912d2e-5004-4cb9-88de-8dcb7bf24701 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:42.740110 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a912d2e-5004-4cb9-88de-8dcb7bf24701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:42.740389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a912d2e-5004-4cb9-88de-8dcb7bf24701 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:42.749296 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a912d2e-5004-4cb9-88de-8dcb7bf24701 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:42.801379 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3a912d2e-5004-4cb9-88de-8dcb7bf24701 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 02 08:56:42.801726 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a912d2e-5004-4cb9-88de-8dcb7bf24701 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:42.812145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 2/4] 10.222.0.100 () {58 vars in 1329 bytes} [Tue Aug 2 08:56:42 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 02 08:56:42.818124 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d6e32-b8e9-4099-9054-6afae79dc984 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:42.833104 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d6e32-b8e9-4099-9054-6afae79dc984 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:42.837541 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1632b-bc30-4a9c-8381-46b6c18a5bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:42.838657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1632b-bc30-4a9c-8381-46b6c18a5bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:42.838950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1632b-bc30-4a9c-8381-46b6c18a5bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:42.839233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1632b-bc30-4a9c-8381-46b6c18a5bd8 admin admin] PATH_INFO: `/v3/projects/1b02845ab43a49ada6db1e8f52cf0026` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:42.840738 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1632b-bc30-4a9c-8381-46b6c18a5bd8 admin admin] RBAC: Authorizing `identity:get_project(project_id=1b02845ab43a49ada6db1e8f52cf0026)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:42.892174 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f6a1632b-bc30-4a9c-8381-46b6c18a5bd8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 02 08:56:42.892531 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1632b-bc30-4a9c-8381-46b6c18a5bd8 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:42.897170 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 3/5] 10.222.0.100 () {56 vars in 1212 bytes} [Tue Aug 2 08:56:42 2022] GET /identity/v3/projects/1b02845ab43a49ada6db1e8f52cf0026 => generated 345 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:56:44.887907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccecbdf-c541-480e-8f22-5aba0363d90c None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:44.889015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccecbdf-c541-480e-8f22-5aba0363d90c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:44.889105 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccecbdf-c541-480e-8f22-5aba0363d90c None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:44.893203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 3/6] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:56:44 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:56:44.898574 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffee293-9464-4142-b9a0-7fe4d35a2c18 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:44.898980 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffee293-9464-4142-b9a0-7fe4d35a2c18 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:44.899134 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffee293-9464-4142-b9a0-7fe4d35a2c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:44.919336 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-3ffee293-9464-4142-b9a0-7fe4d35a2c18 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:56:44.921296 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3ffee293-9464-4142-b9a0-7fe4d35a2c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:56:44.929263 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 4/7] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:56:44 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 02 08:56:44.951884 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccecbdf-c541-480e-8f22-5aba0363d90c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:44.964070 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccecbdf-c541-480e-8f22-5aba0363d90c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:44.968891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa77031-439e-4ffb-804b-8131aa5e1f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:44.969536 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa77031-439e-4ffb-804b-8131aa5e1f09 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:44.969826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa77031-439e-4ffb-804b-8131aa5e1f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:44.970104 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa77031-439e-4ffb-804b-8131aa5e1f09 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:44.977884 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa77031-439e-4ffb-804b-8131aa5e1f09 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:44.979653 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa77031-439e-4ffb-804b-8131aa5e1f09 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:44.988979 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 4/8] 10.222.0.100 () {58 vars in 1329 bytes} [Tue Aug 2 08:56:44 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 02 08:56:44.994211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffee293-9464-4142-b9a0-7fe4d35a2c18 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:45.004172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffee293-9464-4142-b9a0-7fe4d35a2c18 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:45.008532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988bd5b-f9a4-4002-8607-d53fb8177c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:45.009183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988bd5b-f9a4-4002-8607-d53fb8177c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:45.009470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988bd5b-f9a4-4002-8607-d53fb8177c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:45.009746 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988bd5b-f9a4-4002-8607-d53fb8177c2a admin admin] PATH_INFO: `/v3/users/87e4d594c1134b86837b4e4c464d9cfd` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:45.011123 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988bd5b-f9a4-4002-8607-d53fb8177c2a admin admin] RBAC: Authorizing `identity:get_user(user_id=87e4d594c1134b86837b4e4c464d9cfd)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:45.012916 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988bd5b-f9a4-4002-8607-d53fb8177c2a admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:45.014600 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 5/9] 10.222.0.100 () {56 vars in 1203 bytes} [Tue Aug 2 08:56:44 2022] GET /identity/v3/users/87e4d594c1134b86837b4e4c464d9cfd => generated 248 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:56:47.051691 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68757657-8391-48e5-a058-d90c608ec66d None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:47.052208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68757657-8391-48e5-a058-d90c608ec66d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:47.052364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68757657-8391-48e5-a058-d90c608ec66d None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:47.053773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 5/10] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:56:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:56:47.058954 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846c6cc-c1a5-4dba-b7b8-4bb48a2158bd None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:47.059380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846c6cc-c1a5-4dba-b7b8-4bb48a2158bd None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:47.059631 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846c6cc-c1a5-4dba-b7b8-4bb48a2158bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:47.079622 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-f846c6cc-c1a5-4dba-b7b8-4bb48a2158bd None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:56:47.080804 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f846c6cc-c1a5-4dba-b7b8-4bb48a2158bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:56:47.093439 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49652fe1-b76f-4f18-acb8-4be78ccab947 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:47.093723 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49652fe1-b76f-4f18-acb8-4be78ccab947 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:47.094002 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49652fe1-b76f-4f18-acb8-4be78ccab947 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:47.095319 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 6/11] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:56:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:56:47.099983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa601a-89a8-4f13-b897-02d208528936 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:47.100266 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa601a-89a8-4f13-b897-02d208528936 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:47.100540 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa601a-89a8-4f13-b897-02d208528936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:47.143492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG passlib.handlers.bcrypt [None req-88aa601a-89a8-4f13-b897-02d208528936 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80362) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 02 08:56:47.143905 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG passlib.handlers.bcrypt [None req-88aa601a-89a8-4f13-b897-02d208528936 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80362) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 02 08:56:47.162290 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 6/12] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:56:47 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:56:47.184601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846c6cc-c1a5-4dba-b7b8-4bb48a2158bd None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:47.189712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-88aa601a-89a8-4f13-b897-02d208528936 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:56:47.191315 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-88aa601a-89a8-4f13-b897-02d208528936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:56:47.208161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:47.289455 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846c6cc-c1a5-4dba-b7b8-4bb48a2158bd None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:47.294182 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 7/13] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:56:47 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:56:47.315750 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa601a-89a8-4f13-b897-02d208528936 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:47.322381 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc0773-040a-4504-8244-0922e9159b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:47.323028 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc0773-040a-4504-8244-0922e9159b21 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:47.323320 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc0773-040a-4504-8244-0922e9159b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:47.323588 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc0773-040a-4504-8244-0922e9159b21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:47.335470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc0773-040a-4504-8244-0922e9159b21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:56:47.344968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-81dc0773-040a-4504-8244-0922e9159b21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:56:47.344968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:56:47.344968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:56:47.344968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:56:47.344968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:56:47.344968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:56:47.344968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:56:47.344968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:56:47.344968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:56:47.344968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:56:47.344968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:56:47.344968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:56:47.344968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:56:47.344968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:56:47.344968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:56:47.344968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:56:47.345982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:56:47.345982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:56:47.345982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:56:47.345982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:56:47.345982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:56:47.345982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:56:47.345982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:56:47.345982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:56:47.345982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:56:47.345982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:56:47.345982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:56:47.345982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:56:47.345982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:56:47.345982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:56:47.345982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:56:47.345982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:56:47.346826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:56:47.346826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:56:47.346826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:56:47.346826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:56:47.346826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:56:47.346826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:56:47.346826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:56:47.346826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:56:47.346826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:56:47.346826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:56:47.346826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:56:47.346826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:56:47.346826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 7/14] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:56:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:56:47.351964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc0773-040a-4504-8244-0922e9159b21 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:47.417127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa601a-89a8-4f13-b897-02d208528936 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:47.419405 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:47.433822 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:47.453349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc0773-040a-4504-8244-0922e9159b21 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:47.454751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825ba90-6599-43ac-ba35-8683ad2cbefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:47.455646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825ba90-6599-43ac-ba35-8683ad2cbefb None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:47.455935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825ba90-6599-43ac-ba35-8683ad2cbefb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:47.456215 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825ba90-6599-43ac-ba35-8683ad2cbefb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:47.488104 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92efe2a-0f3e-43f3-a48d-43eb2332f3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:47.488718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92efe2a-0f3e-43f3-a48d-43eb2332f3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:47.489007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92efe2a-0f3e-43f3-a48d-43eb2332f3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:47.489317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92efe2a-0f3e-43f3-a48d-43eb2332f3c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:47.490042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92efe2a-0f3e-43f3-a48d-43eb2332f3c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:56:47.490352 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92efe2a-0f3e-43f3-a48d-43eb2332f3c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:47.491752 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92efe2a-0f3e-43f3-a48d-43eb2332f3c8 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:47.505054 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 8/15] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:56:47 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:56:47.510703 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92efe2a-0f3e-43f3-a48d-43eb2332f3c8 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:47.525214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825ba90-6599-43ac-ba35-8683ad2cbefb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:47.526801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825ba90-6599-43ac-ba35-8683ad2cbefb None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:47.631415 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 8/16] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:56:47 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 08:56:47.637506 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825ba90-6599-43ac-ba35-8683ad2cbefb None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:47.641030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:47.721661 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92efe2a-0f3e-43f3-a48d-43eb2332f3c8 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:47.738502 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825ba90-6599-43ac-ba35-8683ad2cbefb None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:47.755735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02dbb6-6b89-413d-a211-c5630a23040e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:47.756386 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02dbb6-6b89-413d-a211-c5630a23040e None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:47.756675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02dbb6-6b89-413d-a211-c5630a23040e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:47.756963 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02dbb6-6b89-413d-a211-c5630a23040e None admin] PATH_INFO: `/v3/users/87e4d594c1134b86837b4e4c464d9cfd` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:47.758176 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02dbb6-6b89-413d-a211-c5630a23040e None admin] RBAC: Authorizing `identity:get_user(user_id=87e4d594c1134b86837b4e4c464d9cfd)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:47.759641 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02dbb6-6b89-413d-a211-c5630a23040e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:47.761269 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 9/17] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:56:47 2022] GET /identity/v3/users/87e4d594c1134b86837b4e4c464d9cfd => generated 248 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:56:47.767067 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02dbb6-6b89-413d-a211-c5630a23040e None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:47.774397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c0190d-5574-4c7b-925c-1de48ef4c25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:47.775278 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0190d-5574-4c7b-925c-1de48ef4c25c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:47.775565 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0190d-5574-4c7b-925c-1de48ef4c25c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:47.775846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0190d-5574-4c7b-925c-1de48ef4c25c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:47.782108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:47.795084 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c0190d-5574-4c7b-925c-1de48ef4c25c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:47.796534 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c0190d-5574-4c7b-925c-1de48ef4c25c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:47.815022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-c1c0190d-5574-4c7b-925c-1de48ef4c25c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:56:47.815022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:56:47.815022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:56:47.815022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:56:47.815022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:56:47.815022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:56:47.815022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:56:47.815022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:56:47.815022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:56:47.815022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:56:47.815022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:56:47.815022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:56:47.815022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:56:47.815022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:56:47.815022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:56:47.815853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:56:47.815853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:56:47.815853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:56:47.815853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:56:47.815853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:56:47.815853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:56:47.815853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:56:47.815853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:56:47.815853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:56:47.815853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:56:47.815853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:56:47.815853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:56:47.815853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:56:47.815853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:56:47.815853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:56:47.815853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:56:47.815853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:56:47.816869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:56:47.816869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:56:47.816869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:56:47.816869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:56:47.816869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 9/18] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:56:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:56:47.821887 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c0190d-5574-4c7b-925c-1de48ef4c25c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:47.869407 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02dbb6-6b89-413d-a211-c5630a23040e None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:47.880560 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:47.906015 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e1f57-d2b3-4c61-853f-194cb2aa195b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:47.906673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e1f57-d2b3-4c61-853f-194cb2aa195b None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:47.906959 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e1f57-d2b3-4c61-853f-194cb2aa195b None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:47.907239 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e1f57-d2b3-4c61-853f-194cb2aa195b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:47.908420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e1f57-d2b3-4c61-853f-194cb2aa195b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:47.909918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e1f57-d2b3-4c61-853f-194cb2aa195b None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:47.911506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 10/19] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:56:47 2022] GET /identity/v3/domains/default => generated 200 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:56:47.917630 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e1f57-d2b3-4c61-853f-194cb2aa195b None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:47.926788 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c0190d-5574-4c7b-925c-1de48ef4c25c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:47.961038 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf3607-4e6a-47de-b82b-8fa31df58d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:47.961682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf3607-4e6a-47de-b82b-8fa31df58d76 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:47.961962 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf3607-4e6a-47de-b82b-8fa31df58d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:47.962236 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf3607-4e6a-47de-b82b-8fa31df58d76 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:47.962924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf3607-4e6a-47de-b82b-8fa31df58d76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:56:47.963229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf3607-4e6a-47de-b82b-8fa31df58d76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:47.964591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf3607-4e6a-47de-b82b-8fa31df58d76 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:47.982320 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 10/20] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:56:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:56:48.002009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:48.023092 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e1f57-d2b3-4c61-853f-194cb2aa195b None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:48.060577 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74f397-1926-471e-97f0-fefa328df914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:48.061483 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74f397-1926-471e-97f0-fefa328df914 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:48.061587 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74f397-1926-471e-97f0-fefa328df914 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:48.061863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74f397-1926-471e-97f0-fefa328df914 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:48.062678 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74f397-1926-471e-97f0-fefa328df914 None admin] RBAC: Adding query filter params (role.id=7cbff51867a7475d837afd51f6c12f11, scope.domain.id=default, user.id=87e4d594c1134b86837b4e4c464d9cfd) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:56:48.062994 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74f397-1926-471e-97f0-fefa328df914 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:48.064498 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74f397-1926-471e-97f0-fefa328df914 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:48.072228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 11/21] 10.222.0.100 () {56 vars in 1327 bytes} [Tue Aug 2 08:56:47 2022] GET /identity/v3/role_assignments?user.id=87e4d594c1134b86837b4e4c464d9cfd&scope.domain.id=default&role.id=7cbff51867a7475d837afd51f6c12f11 => generated 234 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:56:50.451832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026e630-adc7-46f6-89ed-65fa094f0cc2 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:50.452890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026e630-adc7-46f6-89ed-65fa094f0cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:50.453424 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026e630-adc7-46f6-89ed-65fa094f0cc2 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:50.455190 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 11/22] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:56:50 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:56:50.460994 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef67b5-9662-49e6-99a3-105104bfce23 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:50.461991 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef67b5-9662-49e6-99a3-105104bfce23 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:50.462468 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef67b5-9662-49e6-99a3-105104bfce23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:50.486179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-aeef67b5-9662-49e6-99a3-105104bfce23 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:56:50.487570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-aeef67b5-9662-49e6-99a3-105104bfce23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:56:50.578007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 12/23] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:56:50 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:56:50.616423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026e630-adc7-46f6-89ed-65fa094f0cc2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:50.707348 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbc314-8bf4-48aa-9a03-433e6c2fb413 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:50.707986 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbc314-8bf4-48aa-9a03-433e6c2fb413 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:50.708481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbc314-8bf4-48aa-9a03-433e6c2fb413 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:50.710220 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 13/24] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:56:50 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:56:50.721016 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf5af8-8657-4947-8e37-93719eee1859 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:50.721603 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf5af8-8657-4947-8e37-93719eee1859 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:50.722067 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf5af8-8657-4947-8e37-93719eee1859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:50.757699 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-0cbf5af8-8657-4947-8e37-93719eee1859 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:56:50.759280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-0cbf5af8-8657-4947-8e37-93719eee1859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:56:50.770391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026e630-adc7-46f6-89ed-65fa094f0cc2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:50.803203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:50.837665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20727baf-10e8-4e61-8d84-e45a2c9cec11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:50.838335 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20727baf-10e8-4e61-8d84-e45a2c9cec11 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:50.838624 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20727baf-10e8-4e61-8d84-e45a2c9cec11 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:50.838903 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20727baf-10e8-4e61-8d84-e45a2c9cec11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:50.866228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20727baf-10e8-4e61-8d84-e45a2c9cec11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:56:50.873473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-20727baf-10e8-4e61-8d84-e45a2c9cec11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:56:50.873473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:56:50.873473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:56:50.873473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:56:50.873473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:56:50.873473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:56:50.873473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:56:50.873473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:56:50.873473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:56:50.873473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:56:50.873473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:56:50.873473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:56:50.873473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:56:50.873473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:56:50.873473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:56:50.873473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:56:50.874480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:56:50.874480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:56:50.874480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:56:50.874480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:56:50.874480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:56:50.874480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:56:50.874480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:56:50.874480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:56:50.874480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:56:50.874480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:56:50.874480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:56:50.874480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:56:50.874480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:56:50.874480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:56:50.874480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:56:50.874480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:56:50.875226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:56:50.875226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:56:50.875226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:56:50.875226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:56:50.875226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:56:50.875226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:56:50.875226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:56:50.875226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:56:50.875226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:56:50.875226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:56:50.875226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:56:50.875226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:56:50.883569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 12/25] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:56:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:56:50.884727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20727baf-10e8-4e61-8d84-e45a2c9cec11 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:50.912177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 14/26] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:56:50 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:56:50.931638 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2f300-189e-4eb5-a8e0-20e72c7f28c5 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:50.931925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2f300-189e-4eb5-a8e0-20e72c7f28c5 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:50.932210 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2f300-189e-4eb5-a8e0-20e72c7f28c5 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:50.933666 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 15/27] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:56:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:56:50.939532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f0f51-b805-49ef-ad21-c315e67bf847 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:50.940060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f0f51-b805-49ef-ad21-c315e67bf847 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:50.940544 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f0f51-b805-49ef-ad21-c315e67bf847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:50.953825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:50.964748 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-6c9f0f51-b805-49ef-ad21-c315e67bf847 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:56:50.966557 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-6c9f0f51-b805-49ef-ad21-c315e67bf847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:56:51.030297 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20727baf-10e8-4e61-8d84-e45a2c9cec11 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:51.054949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 16/28] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:56:50 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:56:51.056371 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7af5d3-4b89-4a73-b14f-64e95cfd15f7 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:51.056665 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7af5d3-4b89-4a73-b14f-64e95cfd15f7 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:51.056944 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7af5d3-4b89-4a73-b14f-64e95cfd15f7 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:51.058246 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 17/29] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:56:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:56:51.063717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7af5d3-4b89-4a73-b14f-64e95cfd15f7 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:51.098300 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f030c4-0d61-45dd-a801-ae44fa0c55f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:51.099038 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f030c4-0d61-45dd-a801-ae44fa0c55f6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:51.099330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f030c4-0d61-45dd-a801-ae44fa0c55f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:51.099667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f030c4-0d61-45dd-a801-ae44fa0c55f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:51.100533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f030c4-0d61-45dd-a801-ae44fa0c55f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:56:51.100956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f030c4-0d61-45dd-a801-ae44fa0c55f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:51.102474 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f030c4-0d61-45dd-a801-ae44fa0c55f6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:51.116989 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 13/30] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:56:50 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:56:51.118570 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392f0cc-76e0-442f-8115-7f59d73f8b74 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:51.118868 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392f0cc-76e0-442f-8115-7f59d73f8b74 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:51.119300 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392f0cc-76e0-442f-8115-7f59d73f8b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:51.186831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7af5d3-4b89-4a73-b14f-64e95cfd15f7 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:51.226451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c4da0-0532-4449-87d0-0e2e1787d31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:51.227012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c4da0-0532-4449-87d0-0e2e1787d31c None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:51.227296 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c4da0-0532-4449-87d0-0e2e1787d31c None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:51.227573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c4da0-0532-4449-87d0-0e2e1787d31c None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:51.228306 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c4da0-0532-4449-87d0-0e2e1787d31c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:51.229755 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c4da0-0532-4449-87d0-0e2e1787d31c None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:51.234268 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:51.257253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-7392f0cc-76e0-442f-8115-7f59d73f8b74 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:56:51.258478 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7392f0cc-76e0-442f-8115-7f59d73f8b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:56:51.292539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:51.335490 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 14/31] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:56:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:56:51.336470 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392f0cc-76e0-442f-8115-7f59d73f8b74 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:51.382907 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 18/32] 10.222.0.100 () {60 vars in 1136 bytes} [Tue Aug 2 08:56:51 2022] POST /identity/v3/roles => generated 220 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:56:51.385242 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c4da0-0532-4449-87d0-0e2e1787d31c None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:51.456857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392f0cc-76e0-442f-8115-7f59d73f8b74 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:51.502604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c4da0-0532-4449-87d0-0e2e1787d31c None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:51.506807 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee521344-67c2-4193-844a-6ca47b79d4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:51.507530 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee521344-67c2-4193-844a-6ca47b79d4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:51.508054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee521344-67c2-4193-844a-6ca47b79d4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:51.508054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee521344-67c2-4193-844a-6ca47b79d4b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:51.526511 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee521344-67c2-4193-844a-6ca47b79d4b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:51.528779 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee521344-67c2-4193-844a-6ca47b79d4b0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:51.552419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ee521344-67c2-4193-844a-6ca47b79d4b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:56:51.552419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:56:51.552419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:56:51.552419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:56:51.552419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:56:51.552419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:56:51.552419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:56:51.552419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:56:51.552419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:56:51.552419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:56:51.552419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:56:51.552419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:56:51.552419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:56:51.552419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:56:51.552419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:56:51.554340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:56:51.554340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:56:51.554340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:56:51.554340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:56:51.554340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:56:51.554340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:56:51.554340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:56:51.554340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:56:51.554340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:56:51.554340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:56:51.554340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:56:51.554340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:56:51.554340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:56:51.554340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:56:51.554340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:56:51.554340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:56:51.554340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:56:51.556209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:56:51.556209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:56:51.556209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:56:51.556209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:56:51.556209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 15/33] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:56:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:56:51.556209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee521344-67c2-4193-844a-6ca47b79d4b0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:51.563259 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9812b5-0352-4265-be19-52474ac266b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:51.564218 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9812b5-0352-4265-be19-52474ac266b8 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:51.564510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9812b5-0352-4265-be19-52474ac266b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:51.564796 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9812b5-0352-4265-be19-52474ac266b8 None admin] PATH_INFO: `/v3/users/87e4d594c1134b86837b4e4c464d9cfd` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:51.567189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9812b5-0352-4265-be19-52474ac266b8 None admin] RBAC: Authorizing `identity:get_user(user_id=87e4d594c1134b86837b4e4c464d9cfd)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:51.568812 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9812b5-0352-4265-be19-52474ac266b8 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:51.570537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 19/34] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:56:51 2022] GET /identity/v3/users/87e4d594c1134b86837b4e4c464d9cfd => generated 248 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:56:51.572657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0440231-543e-4710-bb40-51f053a02c7c None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:51.572955 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0440231-543e-4710-bb40-51f053a02c7c None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:51.573275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0440231-543e-4710-bb40-51f053a02c7c None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:51.576210 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 20/35] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:56:51 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:56:51.576832 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0440231-543e-4710-bb40-51f053a02c7c None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:51.595159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:51.663866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:51.714013 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee521344-67c2-4193-844a-6ca47b79d4b0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:51.720858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0440231-543e-4710-bb40-51f053a02c7c None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:51.755166 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8fc21-da31-4a83-9398-6cdaab5bf425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:51.755715 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8fc21-da31-4a83-9398-6cdaab5bf425 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:51.756003 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8fc21-da31-4a83-9398-6cdaab5bf425 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:51.756286 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8fc21-da31-4a83-9398-6cdaab5bf425 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:51.757012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8fc21-da31-4a83-9398-6cdaab5bf425 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:51.758443 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8fc21-da31-4a83-9398-6cdaab5bf425 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:51.761065 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d5eb3-23f7-4b12-9c66-6bed37dd5d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:51.761715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d5eb3-23f7-4b12-9c66-6bed37dd5d6a None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:51.762037 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d5eb3-23f7-4b12-9c66-6bed37dd5d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:51.762273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d5eb3-23f7-4b12-9c66-6bed37dd5d6a None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:51.763060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d5eb3-23f7-4b12-9c66-6bed37dd5d6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:56:51.763369 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d5eb3-23f7-4b12-9c66-6bed37dd5d6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:51.764895 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d5eb3-23f7-4b12-9c66-6bed37dd5d6a None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:51.787285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 21/36] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:56:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:56:51.787285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d5eb3-23f7-4b12-9c66-6bed37dd5d6a None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:51.889376 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 16/37] 10.222.0.100 () {60 vars in 1136 bytes} [Tue Aug 2 08:56:51 2022] POST /identity/v3/roles => generated 226 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:56:51.889376 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96dd29-e807-4b69-8795-73ae839e46d2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:51.889376 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96dd29-e807-4b69-8795-73ae839e46d2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:51.889376 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96dd29-e807-4b69-8795-73ae839e46d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:51.906949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3d96dd29-e807-4b69-8795-73ae839e46d2 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:56:51.908670 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3d96dd29-e807-4b69-8795-73ae839e46d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:56:51.908806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d5eb3-23f7-4b12-9c66-6bed37dd5d6a None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:51.943246 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:51.958588 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746471a8-8491-4e9e-acb8-0a384f73390a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:51.959571 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746471a8-8491-4e9e-acb8-0a384f73390a None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:51.960011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746471a8-8491-4e9e-acb8-0a384f73390a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:51.960387 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746471a8-8491-4e9e-acb8-0a384f73390a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:51.962158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746471a8-8491-4e9e-acb8-0a384f73390a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:51.963850 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746471a8-8491-4e9e-acb8-0a384f73390a None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:51.966137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 22/38] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:56:51 2022] GET /identity/v3/domains/default => generated 200 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:56:51.967155 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746471a8-8491-4e9e-acb8-0a384f73390a None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:51.987089 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 17/39] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:56:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:56:51.987947 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96dd29-e807-4b69-8795-73ae839e46d2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:52.001636 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:52.106171 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96dd29-e807-4b69-8795-73ae839e46d2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:52.121254 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746471a8-8491-4e9e-acb8-0a384f73390a None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:52.144709 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b1a14-0575-4a10-af80-eb5e5657e3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:52.145444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b1a14-0575-4a10-af80-eb5e5657e3a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:52.145737 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b1a14-0575-4a10-af80-eb5e5657e3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:52.146026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b1a14-0575-4a10-af80-eb5e5657e3a2 None admin] PATH_INFO: `/v3/domains/default/users/87e4d594c1134b86837b4e4c464d9cfd/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:52.148423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b1a14-0575-4a10-af80-eb5e5657e3a2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=87e4d594c1134b86837b4e4c464d9cfd, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:52.150089 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b1a14-0575-4a10-af80-eb5e5657e3a2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:52.157690 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991e490-1577-4cb3-80ef-90e76be0095c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:52.158397 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991e490-1577-4cb3-80ef-90e76be0095c None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:52.158722 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991e490-1577-4cb3-80ef-90e76be0095c None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:52.159047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991e490-1577-4cb3-80ef-90e76be0095c None admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:52.159874 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991e490-1577-4cb3-80ef-90e76be0095c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:52.161453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991e490-1577-4cb3-80ef-90e76be0095c None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:52.169192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 18/40] 10.222.0.100 () {58 vars in 1366 bytes} [Tue Aug 2 08:56:51 2022] PUT /identity/v3/domains/default/users/87e4d594c1134b86837b4e4c464d9cfd/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:56:52.169192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b1a14-0575-4a10-af80-eb5e5657e3a2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:52.193502 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 23/41] 10.222.0.100 () {60 vars in 1146 bytes} [Tue Aug 2 08:56:51 2022] POST /identity/v3/projects => generated 302 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:56:52.194847 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbf504-7be7-4f6d-90dd-a99eb31e411b None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:52.195138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbf504-7be7-4f6d-90dd-a99eb31e411b None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:52.195416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbf504-7be7-4f6d-90dd-a99eb31e411b None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:52.197099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 24/42] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:56:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:56:52.203137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfa41b-b9eb-47b3-9122-93adddecf68d None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:52.203817 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfa41b-b9eb-47b3-9122-93adddecf68d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:52.204349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfa41b-b9eb-47b3-9122-93adddecf68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:52.242009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-adbfa41b-b9eb-47b3-9122-93adddecf68d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:56:52.244340 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-adbfa41b-b9eb-47b3-9122-93adddecf68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:56:52.283208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:52.298914 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:52.328991 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b1a14-0575-4a10-af80-eb5e5657e3a2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:52.344488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 25/43] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:56:52 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:56:52.370333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfa41b-b9eb-47b3-9122-93adddecf68d None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:52.370650 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382a575-d957-4be7-80cb-9e7949af728a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:52.371265 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382a575-d957-4be7-80cb-9e7949af728a None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:52.371558 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382a575-d957-4be7-80cb-9e7949af728a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:52.371841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382a575-d957-4be7-80cb-9e7949af728a None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:52.373139 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382a575-d957-4be7-80cb-9e7949af728a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:52.374607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382a575-d957-4be7-80cb-9e7949af728a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:52.395355 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 19/44] 10.222.0.100 () {60 vars in 1136 bytes} [Tue Aug 2 08:56:52 2022] POST /identity/v3/roles => generated 224 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:56:52.501246 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfa41b-b9eb-47b3-9122-93adddecf68d None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:52.542172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3281128-d481-45b7-ac48-72ae97c8cf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:52.543682 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3281128-d481-45b7-ac48-72ae97c8cf35 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:52.544247 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3281128-d481-45b7-ac48-72ae97c8cf35 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:52.544757 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3281128-d481-45b7-ac48-72ae97c8cf35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:52.546207 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3281128-d481-45b7-ac48-72ae97c8cf35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:52.547845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3281128-d481-45b7-ac48-72ae97c8cf35 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:52.549780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 26/45] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:56:52 2022] GET /identity/v3/domains/default => generated 200 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:56:52.555870 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382a575-d957-4be7-80cb-9e7949af728a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:52.663809 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382a575-d957-4be7-80cb-9e7949af728a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:52.693987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:52.701683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d25f7-7176-450e-b18f-d064b3294427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:52.702505 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d25f7-7176-450e-b18f-d064b3294427 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:52.702798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d25f7-7176-450e-b18f-d064b3294427 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:52.703083 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d25f7-7176-450e-b18f-d064b3294427 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:52.703785 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d25f7-7176-450e-b18f-d064b3294427 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:52.705270 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d25f7-7176-450e-b18f-d064b3294427 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:52.733858 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 20/46] 10.222.0.100 () {60 vars in 1146 bytes} [Tue Aug 2 08:56:52 2022] POST /identity/v3/projects => generated 313 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:56:54.213312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59383c1-fd90-42d7-bbd1-3dacb7bf1fa8 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:54.213787 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59383c1-fd90-42d7-bbd1-3dacb7bf1fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:54.213787 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59383c1-fd90-42d7-bbd1-3dacb7bf1fa8 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:54.215183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 27/47] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:56:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:56:54.221330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e682fd1-df3e-42d0-a8df-e2817eab597b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:54.221775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e682fd1-df3e-42d0-a8df-e2817eab597b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:54.221897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e682fd1-df3e-42d0-a8df-e2817eab597b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:54.241156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8e682fd1-df3e-42d0-a8df-e2817eab597b None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:56:54.242343 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8e682fd1-df3e-42d0-a8df-e2817eab597b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:56:54.307951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 21/48] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:56:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:56:54.330837 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59383c1-fd90-42d7-bbd1-3dacb7bf1fa8 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:54.373770 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:54.433682 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59383c1-fd90-42d7-bbd1-3dacb7bf1fa8 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:54.467543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267fe87-929c-457e-bf6a-0fb343a80526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:54.468117 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267fe87-929c-457e-bf6a-0fb343a80526 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:54.468404 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267fe87-929c-457e-bf6a-0fb343a80526 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:54.468685 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267fe87-929c-457e-bf6a-0fb343a80526 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:54.478439 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267fe87-929c-457e-bf6a-0fb343a80526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:56:54.483474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-4267fe87-929c-457e-bf6a-0fb343a80526 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:56:54.483474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:56:54.483474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:56:54.483474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:56:54.483474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:56:54.483474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:56:54.483474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:56:54.483474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:56:54.483474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:56:54.483474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:56:54.483474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:56:54.483474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:56:54.483474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:56:54.483474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:56:54.483474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:56:54.483474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:56:54.484474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:56:54.484474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:56:54.484474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:56:54.484474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:56:54.484474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:56:54.484474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:56:54.484474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:56:54.484474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:56:54.484474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:56:54.484474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:56:54.484474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:56:54.484474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:56:54.484474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:56:54.484474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:56:54.484474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:56:54.484474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:56:54.485519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:56:54.485519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:56:54.485519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:56:54.485519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:56:54.485519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:56:54.485519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:56:54.485519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:56:54.485519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:56:54.485519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:56:54.485519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:56:54.485519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:56:54.485519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:56:54.485519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 28/49] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:56:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:56:54.492125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e682fd1-df3e-42d0-a8df-e2817eab597b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:54.591921 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e682fd1-df3e-42d0-a8df-e2817eab597b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:54.605097 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:54.628564 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6271e23-3509-401d-9204-fec42908d725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:54.629219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6271e23-3509-401d-9204-fec42908d725 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:54.629510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6271e23-3509-401d-9204-fec42908d725 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:54.629794 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6271e23-3509-401d-9204-fec42908d725 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:54.630514 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6271e23-3509-401d-9204-fec42908d725 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:56:54.630825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6271e23-3509-401d-9204-fec42908d725 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:54.632216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6271e23-3509-401d-9204-fec42908d725 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:54.641588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 22/50] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:56:54 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:56:54.647075 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267fe87-929c-457e-bf6a-0fb343a80526 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:54.740779 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87574594-6e9a-467f-8d81-b50abebfac2a None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:54.741090 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87574594-6e9a-467f-8d81-b50abebfac2a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:54.741485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87574594-6e9a-467f-8d81-b50abebfac2a None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:54.743021 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 23/51] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:56:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:56:54.745392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:54.748601 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bac28-4816-4643-b762-af5ca5678fc1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:54.748898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bac28-4816-4643-b762-af5ca5678fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:54.749225 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bac28-4816-4643-b762-af5ca5678fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:54.750055 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267fe87-929c-457e-bf6a-0fb343a80526 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:54.768977 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-669bac28-4816-4643-b762-af5ca5678fc1 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:56:54.770201 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-669bac28-4816-4643-b762-af5ca5678fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:56:54.785817 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b5e70-5428-490e-8ad1-dcff1ea70ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:54.786461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b5e70-5428-490e-8ad1-dcff1ea70ecc None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:54.786748 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b5e70-5428-490e-8ad1-dcff1ea70ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:54.787026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b5e70-5428-490e-8ad1-dcff1ea70ecc None admin] PATH_INFO: `/v3/users/87e4d594c1134b86837b4e4c464d9cfd` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:54.788193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b5e70-5428-490e-8ad1-dcff1ea70ecc None admin] RBAC: Authorizing `identity:get_user(user_id=87e4d594c1134b86837b4e4c464d9cfd)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:54.789685 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b5e70-5428-490e-8ad1-dcff1ea70ecc None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:54.791301 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 29/52] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:56:54 2022] GET /identity/v3/users/87e4d594c1134b86837b4e4c464d9cfd => generated 248 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:56:54.796589 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b5e70-5428-490e-8ad1-dcff1ea70ecc None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:54.837834 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 24/53] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:56:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:56:54.860481 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bac28-4816-4643-b762-af5ca5678fc1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:54.895824 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b5e70-5428-490e-8ad1-dcff1ea70ecc None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:54.929727 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c854f-6ebe-43cf-afab-79001bb15afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:54.930445 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c854f-6ebe-43cf-afab-79001bb15afd None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:54.930742 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c854f-6ebe-43cf-afab-79001bb15afd None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:54.931028 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c854f-6ebe-43cf-afab-79001bb15afd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:54.932179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c854f-6ebe-43cf-afab-79001bb15afd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:54.933686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c854f-6ebe-43cf-afab-79001bb15afd None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:54.935337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 30/54] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:56:54 2022] GET /identity/v3/domains/default => generated 200 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:56:54.940927 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c854f-6ebe-43cf-afab-79001bb15afd None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:54.962700 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bac28-4816-4643-b762-af5ca5678fc1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:54.964990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:54.975992 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:54.999736 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be02bf7-39ec-47fd-b301-7429e0de1ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:55.000329 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be02bf7-39ec-47fd-b301-7429e0de1ada None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:55.000617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be02bf7-39ec-47fd-b301-7429e0de1ada None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:55.000897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be02bf7-39ec-47fd-b301-7429e0de1ada None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:55.002100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be02bf7-39ec-47fd-b301-7429e0de1ada None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:55.003798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be02bf7-39ec-47fd-b301-7429e0de1ada None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:55.005430 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 25/55] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:56:54 2022] GET /identity/v3/domains/default => generated 200 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:56:55.011008 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be02bf7-39ec-47fd-b301-7429e0de1ada None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:55.042078 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c854f-6ebe-43cf-afab-79001bb15afd None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:55.074904 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01833c8b-c072-4699-bfcd-330b90939a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:55.076430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01833c8b-c072-4699-bfcd-330b90939a08 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:55.076728 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01833c8b-c072-4699-bfcd-330b90939a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:55.077012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01833c8b-c072-4699-bfcd-330b90939a08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:55.077848 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01833c8b-c072-4699-bfcd-330b90939a08 None admin] RBAC: Adding query filter params (role.id=7cbff51867a7475d837afd51f6c12f11, scope.domain.id=default, user.id=87e4d594c1134b86837b4e4c464d9cfd) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:56:55.078167 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01833c8b-c072-4699-bfcd-330b90939a08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:55.079617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01833c8b-c072-4699-bfcd-330b90939a08 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:55.085287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 31/56] 10.222.0.100 () {56 vars in 1327 bytes} [Tue Aug 2 08:56:54 2022] GET /identity/v3/role_assignments?user.id=87e4d594c1134b86837b4e4c464d9cfd&scope.domain.id=default&role.id=7cbff51867a7475d837afd51f6c12f11 => generated 532 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:56:55.112203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be02bf7-39ec-47fd-b301-7429e0de1ada None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:55.149628 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7124a1af-c591-49d9-b0e3-81ece04c93ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:55.150274 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124a1af-c591-49d9-b0e3-81ece04c93ac None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:55.150564 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124a1af-c591-49d9-b0e3-81ece04c93ac None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:55.150845 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124a1af-c591-49d9-b0e3-81ece04c93ac None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:55.151545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7124a1af-c591-49d9-b0e3-81ece04c93ac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:55.153035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7124a1af-c591-49d9-b0e3-81ece04c93ac None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:55.181460 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 26/57] 10.222.0.100 () {60 vars in 1145 bytes} [Tue Aug 2 08:56:55 2022] POST /identity/v3/projects => generated 299 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 08:56:57.192985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029acb6c-cd8c-4bb0-a64e-3e421d5fe2a5 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:57.193500 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029acb6c-cd8c-4bb0-a64e-3e421d5fe2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:57.193500 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029acb6c-cd8c-4bb0-a64e-3e421d5fe2a5 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:57.194863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 32/58] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:56:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:56:57.200414 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6356df4-3228-44e7-90e1-69fa8a814c27 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:57.200826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6356df4-3228-44e7-90e1-69fa8a814c27 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:57.200984 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6356df4-3228-44e7-90e1-69fa8a814c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:57.219489 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c6356df4-3228-44e7-90e1-69fa8a814c27 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:56:57.220661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c6356df4-3228-44e7-90e1-69fa8a814c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:56:57.237693 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:57.287298 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 27/59] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:56:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:56:57.310765 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029acb6c-cd8c-4bb0-a64e-3e421d5fe2a5 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:57.407792 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:56:57.411785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029acb6c-cd8c-4bb0-a64e-3e421d5fe2a5 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:57.444890 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea5d2b-e92b-4a8a-8205-e29df9d4ddb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:57.445512 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea5d2b-e92b-4a8a-8205-e29df9d4ddb3 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:57.445803 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea5d2b-e92b-4a8a-8205-e29df9d4ddb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:57.446083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea5d2b-e92b-4a8a-8205-e29df9d4ddb3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:57.447218 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea5d2b-e92b-4a8a-8205-e29df9d4ddb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:57.448665 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea5d2b-e92b-4a8a-8205-e29df9d4ddb3 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:57.451056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 33/60] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:56:57 2022] GET /identity/v3/domains/default => generated 200 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:56:57.456489 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6356df4-3228-44e7-90e1-69fa8a814c27 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:56:57.553934 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6356df4-3228-44e7-90e1-69fa8a814c27 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:56:57.587635 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1cfaa-fd6f-415a-b104-890008e10268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:56:57.588260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1cfaa-fd6f-415a-b104-890008e10268 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:56:57.588548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1cfaa-fd6f-415a-b104-890008e10268 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:56:57.588828 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1cfaa-fd6f-415a-b104-890008e10268 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:56:57.589542 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1cfaa-fd6f-415a-b104-890008e10268 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:56:57.592089 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1cfaa-fd6f-415a-b104-890008e10268 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:56:57.619452 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 28/61] 10.222.0.100 () {60 vars in 1137 bytes} [Tue Aug 2 08:56:57 2022] POST /identity/v3/users => generated 276 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:57:00.568109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138287f-85b8-4628-be75-ccbe877a9bb4 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:00.569130 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138287f-85b8-4628-be75-ccbe877a9bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:00.569775 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138287f-85b8-4628-be75-ccbe877a9bb4 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:00.571599 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 34/62] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:00 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:00.578695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5e512-1f55-470e-b14a-2de62f306e6f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:00.579543 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5e512-1f55-470e-b14a-2de62f306e6f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:00.580029 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5e512-1f55-470e-b14a-2de62f306e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:00.607285 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-26d5e512-1f55-470e-b14a-2de62f306e6f None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:00.608995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-26d5e512-1f55-470e-b14a-2de62f306e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:00.618114 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:00.715416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 29/63] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:00 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:00.751982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138287f-85b8-4628-be75-ccbe877a9bb4 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:00.898869 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138287f-85b8-4628-be75-ccbe877a9bb4 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:00.952269 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a393c4-af56-495a-92ab-9f573aca17a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:00.952925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a393c4-af56-495a-92ab-9f573aca17a9 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:00.953240 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a393c4-af56-495a-92ab-9f573aca17a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:00.953525 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a393c4-af56-495a-92ab-9f573aca17a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:00.969479 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d536ab4-e9d4-464e-8cf2-577febe9c0d1 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:00.970074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d536ab4-e9d4-464e-8cf2-577febe9c0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:00.970560 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d536ab4-e9d4-464e-8cf2-577febe9c0d1 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:00.971592 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a393c4-af56-495a-92ab-9f573aca17a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:00.972375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 30/64] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:00 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:57:00.977159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-46a393c4-af56-495a-92ab-9f573aca17a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:00.977159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:00.977159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:00.977159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:00.977159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:00.977159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:00.977159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:00.977159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:00.977159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:00.977159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:00.977159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:00.977159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:00.977159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:00.977159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:00.977159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:00.977159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:00.978011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:00.978011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:00.978011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:00.978011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:00.978011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:00.978011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:00.978011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:00.978011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:00.978011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:00.978011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:00.978011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:00.978011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:00.978011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:00.978011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:00.978011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:00.978011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:00.978779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:00.978779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:00.978779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:00.978779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:00.978779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:00.978779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:00.978779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:00.978779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:00.978779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:00.978779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:00.978779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:00.978779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:00.978779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 35/65] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:00.981883 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979eeeb-3b91-4d43-bb8b-928c5bec5e6a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:00.982389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979eeeb-3b91-4d43-bb8b-928c5bec5e6a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:00.982867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979eeeb-3b91-4d43-bb8b-928c5bec5e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:00.984534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a393c4-af56-495a-92ab-9f573aca17a9 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:01.006831 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b979eeeb-3b91-4d43-bb8b-928c5bec5e6a None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:01.008269 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b979eeeb-3b91-4d43-bb8b-928c5bec5e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:01.029078 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:01.110889 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a393c4-af56-495a-92ab-9f573aca17a9 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:01.120509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 31/66] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:00 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:01.122393 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2254ca-c59d-4930-ae20-282fd52f5220 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:01.122849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2254ca-c59d-4930-ae20-282fd52f5220 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:01.123256 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2254ca-c59d-4930-ae20-282fd52f5220 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:01.124703 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 32/67] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:01.134423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3bf12-5b90-4995-89bd-252f4e50bf43 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:01.135213 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3bf12-5b90-4995-89bd-252f4e50bf43 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:01.136078 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3bf12-5b90-4995-89bd-252f4e50bf43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:01.162553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-bfd3bf12-5b90-4995-89bd-252f4e50bf43 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:01.164336 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bfd3bf12-5b90-4995-89bd-252f4e50bf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:01.168522 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1aa9dd-ac61-4c41-9e27-f9cdaeff7997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:01.169771 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1aa9dd-ac61-4c41-9e27-f9cdaeff7997 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:01.170188 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1aa9dd-ac61-4c41-9e27-f9cdaeff7997 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:01.170483 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1aa9dd-ac61-4c41-9e27-f9cdaeff7997 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:01.171209 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1aa9dd-ac61-4c41-9e27-f9cdaeff7997 None admin] RBAC: Adding query filter params (name=member) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:01.171524 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1aa9dd-ac61-4c41-9e27-f9cdaeff7997 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:01.172938 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1aa9dd-ac61-4c41-9e27-f9cdaeff7997 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:01.184179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 36/68] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:00 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:01.185616 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1aa9dd-ac61-4c41-9e27-f9cdaeff7997 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:01.247718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:01.252898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 33/69] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:57:01.255066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3bf12-5b90-4995-89bd-252f4e50bf43 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:01.297851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:01.309147 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1aa9dd-ac61-4c41-9e27-f9cdaeff7997 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:01.345484 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758aa846-b39c-4d76-9625-b6cacd0a762e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:01.346132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758aa846-b39c-4d76-9625-b6cacd0a762e None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:01.346418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758aa846-b39c-4d76-9625-b6cacd0a762e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:01.346699 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758aa846-b39c-4d76-9625-b6cacd0a762e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:01.357754 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758aa846-b39c-4d76-9625-b6cacd0a762e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:01.363564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-758aa846-b39c-4d76-9625-b6cacd0a762e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:57:01.363564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:01.363564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:01.363564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:01.363564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:01.363564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:01.363564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:01.363564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:01.363564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:01.363564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:01.363564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:01.363564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:01.363564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:01.363564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:01.363564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:01.363564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:01.364559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:01.364559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:01.364559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:01.364559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:01.364559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:01.364559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:01.364559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:01.364559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:01.364559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:01.364559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:01.364559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:01.364559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:01.364559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:01.364559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:01.364559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:01.364559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:01.365546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:01.365546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:01.365546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:01.365546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:01.365546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:01.365546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:01.365546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:01.365546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:01.365546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:01.365546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:01.365546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:57:01.365546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:01.365546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 37/70] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:57:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:01.365546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758aa846-b39c-4d76-9625-b6cacd0a762e None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:01.385839 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3bf12-5b90-4995-89bd-252f4e50bf43 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:01.420327 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2004d0-5e40-4c5f-8ceb-64539b8b9372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:01.420906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2004d0-5e40-4c5f-8ceb-64539b8b9372 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:01.421225 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2004d0-5e40-4c5f-8ceb-64539b8b9372 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:01.421510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2004d0-5e40-4c5f-8ceb-64539b8b9372 None admin] PATH_INFO: `/v3/users/b44229d01ea54d08a04c913b5f878f19` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:01.436819 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2004d0-5e40-4c5f-8ceb-64539b8b9372 None admin] RBAC: Authorizing `identity:get_user(user_id=b44229d01ea54d08a04c913b5f878f19)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:01.438318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2004d0-5e40-4c5f-8ceb-64539b8b9372 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:01.439971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 34/71] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:01 2022] GET /identity/v3/users/b44229d01ea54d08a04c913b5f878f19 => generated 276 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:01.439971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b5b54-cb74-4a6b-9864-369855878a81 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:01.439971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b5b54-cb74-4a6b-9864-369855878a81 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:01.439971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b5b54-cb74-4a6b-9864-369855878a81 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:01.439971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 35/72] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:01 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:01.439971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84b73b-32bc-42c5-8b59-249328e1754e None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:01.444679 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84b73b-32bc-42c5-8b59-249328e1754e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:01.444679 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84b73b-32bc-42c5-8b59-249328e1754e None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:01.448328 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 36/73] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:01 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:01.448328 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84b73b-32bc-42c5-8b59-249328e1754e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:01.472364 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758aa846-b39c-4d76-9625-b6cacd0a762e None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:01.505402 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3039e-bb41-41b1-b997-6ab94d07f7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:01.506049 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3039e-bb41-41b1-b997-6ab94d07f7ee None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:01.506333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3039e-bb41-41b1-b997-6ab94d07f7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:01.506616 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3039e-bb41-41b1-b997-6ab94d07f7ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:01.508319 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:01.517326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3039e-bb41-41b1-b997-6ab94d07f7ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:01.522709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-1cd3039e-bb41-41b1-b997-6ab94d07f7ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:01.522709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:01.522709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:01.522709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:01.522709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:01.522709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:01.522709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:01.522709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:01.522709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:01.522709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:01.522709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:01.522709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:01.522709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:01.522709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:01.522709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:01.522709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:01.523617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:01.523617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:01.523617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:01.523617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:01.523617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:01.523617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:01.523617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:01.523617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:01.523617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:01.523617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:01.523617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:01.523617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:01.523617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:01.523617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:01.523617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:01.523617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:01.530474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:01.530474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:01.530474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:01.530474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:01.530474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:01.530474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:01.530474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:01.530474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:01.530474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:01.530474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:01.530474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:01.530474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:01.530474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 38/74] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:57:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:01.530474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3039e-bb41-41b1-b997-6ab94d07f7ee None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:01.537499 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:01.557104 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84b73b-32bc-42c5-8b59-249328e1754e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:01.594179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d98356-cc6f-45fc-b35b-3a9a2787eba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:01.594807 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d98356-cc6f-45fc-b35b-3a9a2787eba5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:01.595099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d98356-cc6f-45fc-b35b-3a9a2787eba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:01.595384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d98356-cc6f-45fc-b35b-3a9a2787eba5 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:01.596106 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d98356-cc6f-45fc-b35b-3a9a2787eba5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:01.596429 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d98356-cc6f-45fc-b35b-3a9a2787eba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:01.597903 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d98356-cc6f-45fc-b35b-3a9a2787eba5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:01.605956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 37/75] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:01 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:01.611745 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13098e-b390-4d65-914b-a2bb408e5d24 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:01.612040 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13098e-b390-4d65-914b-a2bb408e5d24 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:01.612324 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13098e-b390-4d65-914b-a2bb408e5d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:01.624456 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3039e-bb41-41b1-b997-6ab94d07f7ee None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:01.630595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-7d13098e-b390-4d65-914b-a2bb408e5d24 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:01.631796 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7d13098e-b390-4d65-914b-a2bb408e5d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:01.658006 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa5e11-976e-4a04-a36c-40a1cf61a087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:01.658642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa5e11-976e-4a04-a36c-40a1cf61a087 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:01.658928 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa5e11-976e-4a04-a36c-40a1cf61a087 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:01.659204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa5e11-976e-4a04-a36c-40a1cf61a087 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:01.698216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 38/76] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:01.699716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a4363-f238-4ac6-9337-7573b2968348 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:01.700020 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a4363-f238-4ac6-9337-7573b2968348 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:01.700303 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a4363-f238-4ac6-9337-7573b2968348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:01.717297 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c20a4363-f238-4ac6-9337-7573b2968348 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:01.718470 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c20a4363-f238-4ac6-9337-7573b2968348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:01.742651 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa5e11-976e-4a04-a36c-40a1cf61a087 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:01.744124 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa5e11-976e-4a04-a36c-40a1cf61a087 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:01.755925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:01.764893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:01.787055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 39/77] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:01.790007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a4363-f238-4ac6-9337-7573b2968348 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:01.844397 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 39/78] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:01 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:01.845598 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa5e11-976e-4a04-a36c-40a1cf61a087 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:01.898499 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a4363-f238-4ac6-9337-7573b2968348 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:01.936143 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6193f-68c8-4305-8bd5-a2324fc4ba23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:01.936779 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6193f-68c8-4305-8bd5-a2324fc4ba23 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:01.937067 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6193f-68c8-4305-8bd5-a2324fc4ba23 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:01.937386 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6193f-68c8-4305-8bd5-a2324fc4ba23 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:01.938114 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6193f-68c8-4305-8bd5-a2324fc4ba23 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:01.938429 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6193f-68c8-4305-8bd5-a2324fc4ba23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:01.939804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6193f-68c8-4305-8bd5-a2324fc4ba23 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:01.942733 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa5e11-976e-4a04-a36c-40a1cf61a087 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:01.947693 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 40/79] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:57:01 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:01.948555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6193f-68c8-4305-8bd5-a2324fc4ba23 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:01.956207 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:01.980141 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7d4a7-3384-4fde-a43d-ac5d318668ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:01.980772 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7d4a7-3384-4fde-a43d-ac5d318668ee None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:01.981054 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7d4a7-3384-4fde-a43d-ac5d318668ee None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:01.981895 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7d4a7-3384-4fde-a43d-ac5d318668ee None admin] PATH_INFO: `/v3/users/87e4d594c1134b86837b4e4c464d9cfd` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:01.983048 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7d4a7-3384-4fde-a43d-ac5d318668ee None admin] RBAC: Authorizing `identity:get_user(user_id=87e4d594c1134b86837b4e4c464d9cfd)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:01.984488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7d4a7-3384-4fde-a43d-ac5d318668ee None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:01.986081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 40/80] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:01 2022] GET /identity/v3/users/87e4d594c1134b86837b4e4c464d9cfd => generated 248 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:01.986831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7d4a7-3384-4fde-a43d-ac5d318668ee None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:01.995374 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:02.052049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6193f-68c8-4305-8bd5-a2324fc4ba23 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:02.084949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7d4a7-3384-4fde-a43d-ac5d318668ee None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:02.085247 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8b375-9d11-4a21-a006-41e55ea3c3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:02.085899 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8b375-9d11-4a21-a006-41e55ea3c3ce None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:02.086190 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8b375-9d11-4a21-a006-41e55ea3c3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:02.086473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8b375-9d11-4a21-a006-41e55ea3c3ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:02.087643 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8b375-9d11-4a21-a006-41e55ea3c3ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:02.089099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8b375-9d11-4a21-a006-41e55ea3c3ce None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:02.091260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 41/81] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:01 2022] GET /identity/v3/domains/default => generated 200 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:02.096144 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8b375-9d11-4a21-a006-41e55ea3c3ce None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:02.119521 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f466a-3ee3-4dab-9cdf-83c0dec31500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:02.120169 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f466a-3ee3-4dab-9cdf-83c0dec31500 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:02.120455 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f466a-3ee3-4dab-9cdf-83c0dec31500 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:02.120738 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f466a-3ee3-4dab-9cdf-83c0dec31500 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:02.130990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f466a-3ee3-4dab-9cdf-83c0dec31500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:02.136221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-ff8f466a-3ee3-4dab-9cdf-83c0dec31500 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:02.136221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:02.136221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:02.136221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:02.136221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:02.136221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:02.136221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:02.136221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:02.136221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:02.136221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:02.136221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:02.136221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:02.136221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:02.136221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:02.136221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:02.136221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:02.137063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:02.137063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:02.137063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:02.137063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:02.137063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:02.137063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:02.137063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:02.137063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:02.137063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:02.137063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:02.137063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:02.137063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:02.137063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:02.137063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:02.137063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:02.137063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:02.143353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:02.143353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:02.143353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:02.143353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:02.143353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:02.143353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:02.143353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:02.143353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:02.143353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:02.143353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:02.143353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:02.143353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:02.143353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 41/82] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:02.143353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f466a-3ee3-4dab-9cdf-83c0dec31500 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:02.181159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:02.192390 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8b375-9d11-4a21-a006-41e55ea3c3ce None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:02.203926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:02.227590 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86fd1a-e4b7-4de3-8ce8-3e11e3d809e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:02.228234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86fd1a-e4b7-4de3-8ce8-3e11e3d809e1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:02.228518 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86fd1a-e4b7-4de3-8ce8-3e11e3d809e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:02.228796 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86fd1a-e4b7-4de3-8ce8-3e11e3d809e1 None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:02.229982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86fd1a-e4b7-4de3-8ce8-3e11e3d809e1 None admin] RBAC: Authorizing `identity:get_project(project_id=a78316c83e17485ebfd6d3c10ec19d8e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:02.231568 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86fd1a-e4b7-4de3-8ce8-3e11e3d809e1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:02.233367 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 42/83] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:02 2022] GET /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e => generated 299 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:02.234046 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86fd1a-e4b7-4de3-8ce8-3e11e3d809e1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:02.246851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f466a-3ee3-4dab-9cdf-83c0dec31500 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:02.281240 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4094c7-7755-45a5-a088-0d202bb33008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:02.281876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4094c7-7755-45a5-a088-0d202bb33008 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:02.282163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4094c7-7755-45a5-a088-0d202bb33008 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:02.282440 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4094c7-7755-45a5-a088-0d202bb33008 None admin] PATH_INFO: `/v3/users/b44229d01ea54d08a04c913b5f878f19` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:02.283606 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4094c7-7755-45a5-a088-0d202bb33008 None admin] RBAC: Authorizing `identity:get_user(user_id=b44229d01ea54d08a04c913b5f878f19)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:02.285062 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4094c7-7755-45a5-a088-0d202bb33008 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:02.286664 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 42/84] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:02 2022] GET /identity/v3/users/b44229d01ea54d08a04c913b5f878f19 => generated 276 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:02.287401 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4094c7-7755-45a5-a088-0d202bb33008 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:02.338167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86fd1a-e4b7-4de3-8ce8-3e11e3d809e1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:02.371910 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b4dac-16d9-493c-b4a7-b0cc5ffc1b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:02.372537 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b4dac-16d9-493c-b4a7-b0cc5ffc1b37 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:02.372827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b4dac-16d9-493c-b4a7-b0cc5ffc1b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:02.373121 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b4dac-16d9-493c-b4a7-b0cc5ffc1b37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:02.386431 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4094c7-7755-45a5-a088-0d202bb33008 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:02.393881 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:02.418562 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:02.423934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62612e5-6663-4887-9fcb-064b19c46c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:02.424562 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62612e5-6663-4887-9fcb-064b19c46c2e None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:02.425293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62612e5-6663-4887-9fcb-064b19c46c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:02.425587 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62612e5-6663-4887-9fcb-064b19c46c2e None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:02.426287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62612e5-6663-4887-9fcb-064b19c46c2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:02.427736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62612e5-6663-4887-9fcb-064b19c46c2e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:02.440289 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b4dac-16d9-493c-b4a7-b0cc5ffc1b37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:02.441791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b4dac-16d9-493c-b4a7-b0cc5ffc1b37 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:02.456416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 43/85] 10.222.0.100 () {60 vars in 1137 bytes} [Tue Aug 2 08:57:02 2022] POST /identity/v3/users => generated 290 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:57:02.457289 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62612e5-6663-4887-9fcb-064b19c46c2e None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:02.567911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 43/86] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:02 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:02.567911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b4dac-16d9-493c-b4a7-b0cc5ffc1b37 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:02.581856 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62612e5-6663-4887-9fcb-064b19c46c2e None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:02.623415 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270fb6d-1724-498b-bf5f-7bc87aabbfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:02.624048 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270fb6d-1724-498b-bf5f-7bc87aabbfd0 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:02.624340 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270fb6d-1724-498b-bf5f-7bc87aabbfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:02.624624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270fb6d-1724-498b-bf5f-7bc87aabbfd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:02.625432 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270fb6d-1724-498b-bf5f-7bc87aabbfd0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:02.625745 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270fb6d-1724-498b-bf5f-7bc87aabbfd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:02.627144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270fb6d-1724-498b-bf5f-7bc87aabbfd0 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:02.636722 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 44/87] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:02 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:02.637691 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270fb6d-1724-498b-bf5f-7bc87aabbfd0 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:02.659576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:02.670155 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:02.687528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b4dac-16d9-493c-b4a7-b0cc5ffc1b37 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:02.733211 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11d94c-4624-4960-904c-f314e56473b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:02.733778 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11d94c-4624-4960-904c-f314e56473b9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:02.734065 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11d94c-4624-4960-904c-f314e56473b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:02.734344 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11d94c-4624-4960-904c-f314e56473b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:02.735150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11d94c-4624-4960-904c-f314e56473b9 None admin] RBAC: Adding query filter params (role.id=f739a9c857cd4050896a6814c8bc0120, scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e, user.id=b44229d01ea54d08a04c913b5f878f19) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:02.735465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11d94c-4624-4960-904c-f314e56473b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:02.736922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11d94c-4624-4960-904c-f314e56473b9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:02.748186 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 44/88] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:02 2022] GET /identity/v3/role_assignments?user.id=b44229d01ea54d08a04c913b5f878f19&scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e&role.id=f739a9c857cd4050896a6814c8bc0120 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:02.748920 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11d94c-4624-4960-904c-f314e56473b9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:02.773073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270fb6d-1724-498b-bf5f-7bc87aabbfd0 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:02.827719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267f9c7-1d62-4c70-ade6-4ed539dbfe87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:02.828358 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267f9c7-1d62-4c70-ade6-4ed539dbfe87 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:02.828646 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267f9c7-1d62-4c70-ade6-4ed539dbfe87 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:02.828926 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267f9c7-1d62-4c70-ade6-4ed539dbfe87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:02.902148 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11d94c-4624-4960-904c-f314e56473b9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:02.946482 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b6219-23a1-4949-bdd5-499f1188ba75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:02.947089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3267f9c7-1d62-4c70-ade6-4ed539dbfe87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:02.947199 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b6219-23a1-4949-bdd5-499f1188ba75 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:02.947435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b6219-23a1-4949-bdd5-499f1188ba75 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:02.947724 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b6219-23a1-4949-bdd5-499f1188ba75 None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:02.948639 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3267f9c7-1d62-4c70-ade6-4ed539dbfe87 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:02.949549 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b6219-23a1-4949-bdd5-499f1188ba75 None admin] RBAC: Authorizing `identity:get_project(project_id=a78316c83e17485ebfd6d3c10ec19d8e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:02.952236 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b6219-23a1-4949-bdd5-499f1188ba75 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:02.953990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 45/89] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:02 2022] GET /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e => generated 299 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:02.960157 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b6219-23a1-4949-bdd5-499f1188ba75 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:02.974855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:02.978900 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:03.071212 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b6219-23a1-4949-bdd5-499f1188ba75 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:03.117013 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10043886-be1a-4bd6-841e-9708c417c64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:03.117711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10043886-be1a-4bd6-841e-9708c417c64b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:03.118002 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10043886-be1a-4bd6-841e-9708c417c64b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:03.118287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10043886-be1a-4bd6-841e-9708c417c64b None admin] PATH_INFO: `/v3/users/b44229d01ea54d08a04c913b5f878f19` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:03.120180 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10043886-be1a-4bd6-841e-9708c417c64b None admin] RBAC: Authorizing `identity:get_user(user_id=b44229d01ea54d08a04c913b5f878f19)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:03.123072 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10043886-be1a-4bd6-841e-9708c417c64b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:03.124902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 46/90] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:02 2022] GET /identity/v3/users/b44229d01ea54d08a04c913b5f878f19 => generated 276 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:03.125723 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10043886-be1a-4bd6-841e-9708c417c64b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:03.140680 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 45/91] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:02 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:03.141581 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267f9c7-1d62-4c70-ade6-4ed539dbfe87 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:03.257145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:03.294922 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267f9c7-1d62-4c70-ade6-4ed539dbfe87 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:03.295937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10043886-be1a-4bd6-841e-9708c417c64b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:03.337163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:03.357566 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6113c-7d11-4a3d-a68e-629b754256dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:03.358206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6113c-7d11-4a3d-a68e-629b754256dd None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:03.358497 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6113c-7d11-4a3d-a68e-629b754256dd None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:03.358779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6113c-7d11-4a3d-a68e-629b754256dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:03.361691 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04783955-2373-4a7b-92e8-c677c5342cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:03.362320 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04783955-2373-4a7b-92e8-c677c5342cec None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:03.362610 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04783955-2373-4a7b-92e8-c677c5342cec None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:03.362891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04783955-2373-4a7b-92e8-c677c5342cec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:03.363692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04783955-2373-4a7b-92e8-c677c5342cec None admin] RBAC: Adding query filter params (role.id=7cbff51867a7475d837afd51f6c12f11, scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e, user.id=87e4d594c1134b86837b4e4c464d9cfd) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:03.364010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04783955-2373-4a7b-92e8-c677c5342cec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:03.365482 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04783955-2373-4a7b-92e8-c677c5342cec None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:03.376191 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 47/92] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:03 2022] GET /identity/v3/role_assignments?user.id=87e4d594c1134b86837b4e4c464d9cfd&scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e&role.id=7cbff51867a7475d837afd51f6c12f11 => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:03.377031 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04783955-2373-4a7b-92e8-c677c5342cec None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:03.458824 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb6113c-7d11-4a3d-a68e-629b754256dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:03.460331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb6113c-7d11-4a3d-a68e-629b754256dd None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:03.553212 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04783955-2373-4a7b-92e8-c677c5342cec None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:03.593231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98767f7d-7792-4200-b87a-bcd966180408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:03.593835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98767f7d-7792-4200-b87a-bcd966180408 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:03.594125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98767f7d-7792-4200-b87a-bcd966180408 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:03.594523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98767f7d-7792-4200-b87a-bcd966180408 None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:03.595739 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98767f7d-7792-4200-b87a-bcd966180408 None admin] RBAC: Authorizing `identity:get_project(project_id=a78316c83e17485ebfd6d3c10ec19d8e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:03.597249 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98767f7d-7792-4200-b87a-bcd966180408 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:03.598979 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 48/93] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:03 2022] GET /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e => generated 299 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:03.605162 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98767f7d-7792-4200-b87a-bcd966180408 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:03.619251 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:03.658021 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 46/94] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:03 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:03.664056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6113c-7d11-4a3d-a68e-629b754256dd None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:03.691542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:03.724489 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98767f7d-7792-4200-b87a-bcd966180408 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:03.769135 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb44ad5c-a335-4130-9048-1056a363a40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:03.769135 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44ad5c-a335-4130-9048-1056a363a40a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:03.769424 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44ad5c-a335-4130-9048-1056a363a40a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:03.773169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44ad5c-a335-4130-9048-1056a363a40a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:03.773169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb44ad5c-a335-4130-9048-1056a363a40a None admin] RBAC: Adding query filter params (role.id=bb986e49c38c40bab629ff32371b9358, scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e, user.id=b44229d01ea54d08a04c913b5f878f19) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:03.773169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb44ad5c-a335-4130-9048-1056a363a40a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:03.773169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb44ad5c-a335-4130-9048-1056a363a40a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:03.793150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 49/95] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:03 2022] GET /identity/v3/role_assignments?user.id=b44229d01ea54d08a04c913b5f878f19&scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e&role.id=bb986e49c38c40bab629ff32371b9358 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:03.821722 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6113c-7d11-4a3d-a68e-629b754256dd None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:03.882059 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab652c-26f3-4491-9124-7b0d3a713587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:03.885135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab652c-26f3-4491-9124-7b0d3a713587 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:03.885135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab652c-26f3-4491-9124-7b0d3a713587 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:03.885135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab652c-26f3-4491-9124-7b0d3a713587 None admin] PATH_INFO: `/v3/projects/dfeb4e06d7754aa3940fc70d5d408a30` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:03.885135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab652c-26f3-4491-9124-7b0d3a713587 None admin] RBAC: Authorizing `identity:get_project(project_id=dfeb4e06d7754aa3940fc70d5d408a30)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:03.886994 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab652c-26f3-4491-9124-7b0d3a713587 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:03.894766 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 47/96] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:03 2022] GET /identity/v3/projects/dfeb4e06d7754aa3940fc70d5d408a30 => generated 313 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:03.912541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb44ad5c-a335-4130-9048-1056a363a40a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:04.036365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:04.063899 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb44ad5c-a335-4130-9048-1056a363a40a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:04.102089 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2506df-8508-473d-bb29-bb49c6fc42f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:04.102747 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2506df-8508-473d-bb29-bb49c6fc42f1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:04.103026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2506df-8508-473d-bb29-bb49c6fc42f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:04.103311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2506df-8508-473d-bb29-bb49c6fc42f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:04.104118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2506df-8508-473d-bb29-bb49c6fc42f1 None admin] RBAC: Adding query filter params (role.id=f739a9c857cd4050896a6814c8bc0120, scope.project.id=dfeb4e06d7754aa3940fc70d5d408a30, user.id=b44229d01ea54d08a04c913b5f878f19) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:04.104439 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2506df-8508-473d-bb29-bb49c6fc42f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:04.105907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2506df-8508-473d-bb29-bb49c6fc42f1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:04.112433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 50/97] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:03 2022] GET /identity/v3/role_assignments?user.id=b44229d01ea54d08a04c913b5f878f19&scope.project.id=dfeb4e06d7754aa3940fc70d5d408a30&role.id=f739a9c857cd4050896a6814c8bc0120 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:05.416001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326d1bd-ba7b-47a6-a4e2-b6030e39f48d None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:05.416001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326d1bd-ba7b-47a6-a4e2-b6030e39f48d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:05.416001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326d1bd-ba7b-47a6-a4e2-b6030e39f48d None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:05.416001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 48/98] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:05.422309 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a271a35-5bb4-4d39-8545-b2e22b8fd3a9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:05.422696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a271a35-5bb4-4d39-8545-b2e22b8fd3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:05.422894 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a271a35-5bb4-4d39-8545-b2e22b8fd3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:05.448110 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3a271a35-5bb4-4d39-8545-b2e22b8fd3a9 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:05.449362 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3a271a35-5bb4-4d39-8545-b2e22b8fd3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:05.543754 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 51/99] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:05.584812 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326d1bd-ba7b-47a6-a4e2-b6030e39f48d None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:05.711481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:05.727430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326d1bd-ba7b-47a6-a4e2-b6030e39f48d None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:05.753656 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2573b-e814-4977-b626-82ed7b105306 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:05.753991 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2573b-e814-4977-b626-82ed7b105306 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:05.754234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2573b-e814-4977-b626-82ed7b105306 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:05.755616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 52/100] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:05.762532 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3ec6f-91b2-4984-926e-f1b7e436d4ff None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:05.762830 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3ec6f-91b2-4984-926e-f1b7e436d4ff None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:05.763136 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3ec6f-91b2-4984-926e-f1b7e436d4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:05.783417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4be07-9ab9-4ce8-a507-9a795bfaf5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:05.784431 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4be07-9ab9-4ce8-a507-9a795bfaf5ce None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:05.784930 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4be07-9ab9-4ce8-a507-9a795bfaf5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:05.785449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4be07-9ab9-4ce8-a507-9a795bfaf5ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:05.798303 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f3d3ec6f-91b2-4984-926e-f1b7e436d4ff None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:05.799518 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f3d3ec6f-91b2-4984-926e-f1b7e436d4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:05.803969 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d4be07-9ab9-4ce8-a507-9a795bfaf5ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:05.810172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-23d4be07-9ab9-4ce8-a507-9a795bfaf5ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:57:05.810172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:05.810172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:05.810172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:05.810172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:05.810172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:05.810172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:05.810172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:05.810172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:05.810172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:05.810172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:05.810172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:05.810172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:05.810172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:05.810172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:05.810172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:05.811005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:05.811005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:05.811005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:05.811005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:05.811005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:05.811005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:05.811005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:05.811005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:05.811005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:05.811005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:05.811005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:05.811005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:05.811005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:05.811005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:05.811005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:05.811005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:05.811768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:05.811768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:05.811768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:05.811768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:05.811768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:05.811768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:05.811768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:05.811768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:05.811768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:05.811768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:05.811768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:57:05.811768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:05.814113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 49/101] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:05.820830 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4be07-9ab9-4ce8-a507-9a795bfaf5ce None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:05.948256 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:05.966272 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 53/102] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:57:05.997595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4be07-9ab9-4ce8-a507-9a795bfaf5ce None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:06.011393 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3ec6f-91b2-4984-926e-f1b7e436d4ff None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:06.074635 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28387b33-d9b8-45da-a9f2-8281233a9472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:06.075803 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28387b33-d9b8-45da-a9f2-8281233a9472 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:06.076267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28387b33-d9b8-45da-a9f2-8281233a9472 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:06.076549 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28387b33-d9b8-45da-a9f2-8281233a9472 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:06.077492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28387b33-d9b8-45da-a9f2-8281233a9472 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:06.077806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28387b33-d9b8-45da-a9f2-8281233a9472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:06.082043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28387b33-d9b8-45da-a9f2-8281233a9472 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:06.104219 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 50/103] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:05 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:06.111900 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28387b33-d9b8-45da-a9f2-8281233a9472 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:06.150644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:06.229753 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3ec6f-91b2-4984-926e-f1b7e436d4ff None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:06.299148 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aeccea-fb0f-44c2-bdaa-5cf4254d3439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:06.299858 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aeccea-fb0f-44c2-bdaa-5cf4254d3439 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:06.300148 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aeccea-fb0f-44c2-bdaa-5cf4254d3439 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:06.300428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aeccea-fb0f-44c2-bdaa-5cf4254d3439 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:06.302213 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aeccea-fb0f-44c2-bdaa-5cf4254d3439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:06.303658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aeccea-fb0f-44c2-bdaa-5cf4254d3439 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:06.305283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 54/104] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:06 2022] GET /identity/v3/domains/default => generated 200 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:06.311156 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28387b33-d9b8-45da-a9f2-8281233a9472 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:06.312684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aeccea-fb0f-44c2-bdaa-5cf4254d3439 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:06.378862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8b917-bfb3-46bf-a4ab-c75b8411fc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:06.379808 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8b917-bfb3-46bf-a4ab-c75b8411fc0e None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:06.381558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8b917-bfb3-46bf-a4ab-c75b8411fc0e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:06.381980 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8b917-bfb3-46bf-a4ab-c75b8411fc0e None admin] PATH_INFO: `/v3/users/a716df57e0834669a75c53f672b102ea` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:06.408842 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8b917-bfb3-46bf-a4ab-c75b8411fc0e None admin] RBAC: Authorizing `identity:get_user(user_id=a716df57e0834669a75c53f672b102ea)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:06.412009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8b917-bfb3-46bf-a4ab-c75b8411fc0e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:06.414059 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 51/105] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:06 2022] GET /identity/v3/users/a716df57e0834669a75c53f672b102ea => generated 290 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:06.420649 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8b917-bfb3-46bf-a4ab-c75b8411fc0e None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:06.424733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:06.493450 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aeccea-fb0f-44c2-bdaa-5cf4254d3439 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:06.550296 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:06.557348 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228eb0c9-5506-4560-8af8-7133aa4c6aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:06.558000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228eb0c9-5506-4560-8af8-7133aa4c6aa0 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:06.558312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228eb0c9-5506-4560-8af8-7133aa4c6aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:06.558646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228eb0c9-5506-4560-8af8-7133aa4c6aa0 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:06.559910 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228eb0c9-5506-4560-8af8-7133aa4c6aa0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:06.561531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228eb0c9-5506-4560-8af8-7133aa4c6aa0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:06.584153 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8b917-bfb3-46bf-a4ab-c75b8411fc0e None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:06.625211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4d168-d08b-4151-9fd4-b65bc9133f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:06.626105 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4d168-d08b-4151-9fd4-b65bc9133f79 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:06.626534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4d168-d08b-4151-9fd4-b65bc9133f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:06.627010 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4d168-d08b-4151-9fd4-b65bc9133f79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:06.635043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 55/106] 10.222.0.100 () {60 vars in 1146 bytes} [Tue Aug 2 08:57:06 2022] POST /identity/v3/projects => generated 303 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:57:06.636363 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1af7b-7a2e-46a6-b02d-2adb85bea36d None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:06.636657 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1af7b-7a2e-46a6-b02d-2adb85bea36d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:06.636937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1af7b-7a2e-46a6-b02d-2adb85bea36d None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:06.638974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 56/107] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:06 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:06.647610 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b412c-d61c-4c40-b543-83ed960be187 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:06.647897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b412c-d61c-4c40-b543-83ed960be187 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:06.648182 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b412c-d61c-4c40-b543-83ed960be187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:06.686238 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-2c5b412c-d61c-4c40-b543-83ed960be187 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:06.687484 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2c5b412c-d61c-4c40-b543-83ed960be187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:06.750453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4d168-d08b-4151-9fd4-b65bc9133f79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:06.751880 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4d168-d08b-4151-9fd4-b65bc9133f79 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:06.804031 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 57/108] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:06 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:57:06.811562 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ca030-ba13-47e4-a477-cc3f22443112 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:06.812043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ca030-ba13-47e4-a477-cc3f22443112 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:06.812526 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ca030-ba13-47e4-a477-cc3f22443112 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:06.814239 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 58/109] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:06 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:06.815469 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9da5e3-bbf3-4a77-aaeb-0866692097ee None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:06.815932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9da5e3-bbf3-4a77-aaeb-0866692097ee None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:06.816485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9da5e3-bbf3-4a77-aaeb-0866692097ee None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:06.818132 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 59/110] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:06 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:06.823615 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be6615-8040-4f40-ba52-d2d329bc8391 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:06.824156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be6615-8040-4f40-ba52-d2d329bc8391 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:06.825317 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be6615-8040-4f40-ba52-d2d329bc8391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:06.838802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:06.847578 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-87be6615-8040-4f40-ba52-d2d329bc8391 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:06.849150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-87be6615-8040-4f40-ba52-d2d329bc8391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:06.876600 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 52/111] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:06 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:06.877952 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06e0d8-8b1e-4d47-aa84-b2ce764371c2 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:06.878367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06e0d8-8b1e-4d47-aa84-b2ce764371c2 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:06.878749 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06e0d8-8b1e-4d47-aa84-b2ce764371c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:06.899172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-ad06e0d8-8b1e-4d47-aa84-b2ce764371c2 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:06.900236 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ad06e0d8-8b1e-4d47-aa84-b2ce764371c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:06.905370 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:06.939257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 60/112] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:06 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:57:06.940253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be6615-8040-4f40-ba52-d2d329bc8391 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:06.994031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 53/113] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:06 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:06.994902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06e0d8-8b1e-4d47-aa84-b2ce764371c2 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:07.062398 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be6615-8040-4f40-ba52-d2d329bc8391 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:07.100751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b99b72-467b-4ee1-be63-a9d830b23297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:07.101599 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b99b72-467b-4ee1-be63-a9d830b23297 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:07.101884 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b99b72-467b-4ee1-be63-a9d830b23297 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:07.102482 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b99b72-467b-4ee1-be63-a9d830b23297 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:07.110711 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06e0d8-8b1e-4d47-aa84-b2ce764371c2 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:07.112394 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:07.117069 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b99b72-467b-4ee1-be63-a9d830b23297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:07.123955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-04b99b72-467b-4ee1-be63-a9d830b23297 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:07.123955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:07.123955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:07.123955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:07.123955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:07.123955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:07.123955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:07.123955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:07.123955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:07.123955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:07.123955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:07.123955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:07.123955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:07.123955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:07.123955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:07.123955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:07.124880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:07.124880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:07.124880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:07.124880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:07.124880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:07.124880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:07.124880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:07.124880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:07.124880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:07.124880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:07.124880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:07.124880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:07.124880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:07.124880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:07.124880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:07.124880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:07.130992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:07.130992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:07.130992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:07.130992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:07.130992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:07.130992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:07.130992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:07.130992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:07.130992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:07.130992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:07.130992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:07.130992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:07.130992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 61/114] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:07.130992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b99b72-467b-4ee1-be63-a9d830b23297 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:07.150777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8a197-441a-4fa3-b2e0-c08781add131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:07.151475 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8a197-441a-4fa3-b2e0-c08781add131 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:07.151757 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8a197-441a-4fa3-b2e0-c08781add131 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:07.152031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8a197-441a-4fa3-b2e0-c08781add131 None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:07.153346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb8a197-441a-4fa3-b2e0-c08781add131 None admin] RBAC: Authorizing `identity:get_project(project_id=a78316c83e17485ebfd6d3c10ec19d8e)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:07.154832 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb8a197-441a-4fa3-b2e0-c08781add131 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:07.156710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 54/115] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:06 2022] GET /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e => generated 299 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:07.158027 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a0b8c-1d54-4e68-96cb-60f450e892fb None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:07.158314 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a0b8c-1d54-4e68-96cb-60f450e892fb None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:07.158592 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a0b8c-1d54-4e68-96cb-60f450e892fb None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:07.159790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 55/116] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:07 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:07.160638 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a0b8c-1d54-4e68-96cb-60f450e892fb None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:07.181104 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:07.249267 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b99b72-467b-4ee1-be63-a9d830b23297 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:07.276726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a0b8c-1d54-4e68-96cb-60f450e892fb None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:07.285306 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46abcdb-0cef-4918-a66e-d1d5e7118e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:07.286040 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46abcdb-0cef-4918-a66e-d1d5e7118e91 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:07.286419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46abcdb-0cef-4918-a66e-d1d5e7118e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:07.286768 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46abcdb-0cef-4918-a66e-d1d5e7118e91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:07.297852 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46abcdb-0cef-4918-a66e-d1d5e7118e91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:07.303000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-e46abcdb-0cef-4918-a66e-d1d5e7118e91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:57:07.303000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:07.303000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:07.303000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:07.303000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:07.303000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:07.303000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:07.303000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:07.303000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:07.303000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:07.303000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:07.303000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:07.303000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:07.303000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:07.303000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:07.303000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:07.303874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:07.303874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:07.303874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:07.303874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:07.303874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:07.303874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:07.303874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:07.303874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:07.303874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:07.303874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:07.303874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:07.303874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:07.303874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:07.303874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:07.303874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:07.303874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:07.304926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:07.304926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:07.304926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:07.304926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:07.304926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:07.304926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:07.304926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:07.304926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:07.304926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:07.304926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:07.304926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:57:07.304926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:07.304926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 62/117] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:57:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:07.309516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46abcdb-0cef-4918-a66e-d1d5e7118e91 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:07.316688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce8e73-5d57-4f29-bc79-f51b3586d9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:07.317362 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce8e73-5d57-4f29-bc79-f51b3586d9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:07.317709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce8e73-5d57-4f29-bc79-f51b3586d9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:07.317992 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce8e73-5d57-4f29-bc79-f51b3586d9c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:07.328452 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce8e73-5d57-4f29-bc79-f51b3586d9c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:07.333465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-e5ce8e73-5d57-4f29-bc79-f51b3586d9c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:07.333465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:07.333465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:07.333465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:07.333465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:07.333465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:07.333465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:07.333465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:07.333465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:07.333465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:07.333465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:07.333465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:07.333465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:07.333465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:07.333465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:07.333465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:07.334470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:07.334470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:07.334470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:07.334470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:07.334470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:07.334470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:07.334470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:07.334470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:07.334470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:07.334470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:07.334470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:07.334470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:07.334470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:07.334470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:07.334470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:07.334470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:07.335331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:07.335331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:07.335331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:07.335331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:07.335331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:07.335331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:07.335331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:07.335331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:07.335331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:07.335331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:07.335331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:07.335331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:07.335331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 56/118] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:57:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:07.338260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce8e73-5d57-4f29-bc79-f51b3586d9c1 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:07.380787 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:07.416570 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:07.420869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46abcdb-0cef-4918-a66e-d1d5e7118e91 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:07.458639 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce8e73-5d57-4f29-bc79-f51b3586d9c1 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:07.459189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc047282-f70c-4107-a607-6c98897aca12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:07.459778 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc047282-f70c-4107-a607-6c98897aca12 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:07.460062 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc047282-f70c-4107-a607-6c98897aca12 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:07.460402 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc047282-f70c-4107-a607-6c98897aca12 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:07.461053 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc047282-f70c-4107-a607-6c98897aca12 None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:07.462121 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc047282-f70c-4107-a607-6c98897aca12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:07.462854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc047282-f70c-4107-a607-6c98897aca12 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:07.472336 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 63/119] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:07 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:07.473417 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3ad87-678d-460c-9760-6deac366662b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:07.473709 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3ad87-678d-460c-9760-6deac366662b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:07.473988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3ad87-678d-460c-9760-6deac366662b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:07.491955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-83a3ad87-678d-460c-9760-6deac366662b None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:07.493152 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-83a3ad87-678d-460c-9760-6deac366662b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:07.500274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e66278-1015-4e73-8e6e-768d33576218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:07.500905 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e66278-1015-4e73-8e6e-768d33576218 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:07.501222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e66278-1015-4e73-8e6e-768d33576218 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:07.501509 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e66278-1015-4e73-8e6e-768d33576218 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:07.502304 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e66278-1015-4e73-8e6e-768d33576218 None admin] RBAC: Adding query filter params (role.id=8724fd98145549928aa27d32c24e610a, scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e, user.id=a716df57e0834669a75c53f672b102ea) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:07.502885 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e66278-1015-4e73-8e6e-768d33576218 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:07.504355 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e66278-1015-4e73-8e6e-768d33576218 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:07.510163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 57/120] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:07 2022] GET /identity/v3/role_assignments?user.id=a716df57e0834669a75c53f672b102ea&scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e&role.id=8724fd98145549928aa27d32c24e610a => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:07.511049 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e66278-1015-4e73-8e6e-768d33576218 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:07.591833 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 64/121] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:07.592762 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3ad87-678d-460c-9760-6deac366662b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:07.654638 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:07.663704 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e66278-1015-4e73-8e6e-768d33576218 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:07.704967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39878ccc-11b1-4321-9d52-d67fe2fb4536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:07.705651 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39878ccc-11b1-4321-9d52-d67fe2fb4536 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:07.705987 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39878ccc-11b1-4321-9d52-d67fe2fb4536 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:07.706276 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39878ccc-11b1-4321-9d52-d67fe2fb4536 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:07.707273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39878ccc-11b1-4321-9d52-d67fe2fb4536 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:07.707397 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39878ccc-11b1-4321-9d52-d67fe2fb4536 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:07.708745 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39878ccc-11b1-4321-9d52-d67fe2fb4536 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:07.719756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 58/122] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:07 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:07.720756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39878ccc-11b1-4321-9d52-d67fe2fb4536 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:07.729477 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:07.734006 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3ad87-678d-460c-9760-6deac366662b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:07.778589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add93a7-27db-4020-920a-bd5fc53cf20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:07.779893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add93a7-27db-4020-920a-bd5fc53cf20e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:07.780328 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add93a7-27db-4020-920a-bd5fc53cf20e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:07.781284 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add93a7-27db-4020-920a-bd5fc53cf20e None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:07.782328 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add93a7-27db-4020-920a-bd5fc53cf20e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:07.782795 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add93a7-27db-4020-920a-bd5fc53cf20e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:07.784845 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add93a7-27db-4020-920a-bd5fc53cf20e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:07.796399 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 65/123] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:57:07 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:07.802673 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add93a7-27db-4020-920a-bd5fc53cf20e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:07.834215 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39878ccc-11b1-4321-9d52-d67fe2fb4536 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:07.875494 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f30abc-4757-47b0-9b88-cd0782b4d896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:07.876175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f30abc-4757-47b0-9b88-cd0782b4d896 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:07.876469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f30abc-4757-47b0-9b88-cd0782b4d896 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:07.876754 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f30abc-4757-47b0-9b88-cd0782b4d896 None admin] PATH_INFO: `/v3/users/b44229d01ea54d08a04c913b5f878f19` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:07.885294 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f30abc-4757-47b0-9b88-cd0782b4d896 None admin] RBAC: Authorizing `identity:get_user(user_id=b44229d01ea54d08a04c913b5f878f19)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:07.885294 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f30abc-4757-47b0-9b88-cd0782b4d896 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:07.885294 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 59/124] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:07 2022] GET /identity/v3/users/b44229d01ea54d08a04c913b5f878f19 => generated 276 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:07.887216 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f30abc-4757-47b0-9b88-cd0782b4d896 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:07.925679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:07.957646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add93a7-27db-4020-920a-bd5fc53cf20e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:07.997779 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182645af-f2de-4157-8216-f2d4879598f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:07.998664 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182645af-f2de-4157-8216-f2d4879598f0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:07.999089 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182645af-f2de-4157-8216-f2d4879598f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:07.999503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182645af-f2de-4157-8216-f2d4879598f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:08.015628 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182645af-f2de-4157-8216-f2d4879598f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:08.020209 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f30abc-4757-47b0-9b88-cd0782b4d896 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:08.023563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-182645af-f2de-4157-8216-f2d4879598f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:08.023563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:08.023563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:08.023563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:08.023563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:08.023563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:08.023563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:08.023563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:08.023563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:08.023563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:08.023563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:08.023563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:08.023563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:08.023563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:08.023563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:08.023563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:08.024433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:08.024433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:08.024433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:08.024433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:08.024433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:08.024433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:08.024433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:08.024433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:08.024433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:08.024433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:08.024433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:08.024433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:08.024433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:08.024433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:08.024433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:08.024433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:08.025261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:08.025261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:08.025261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:08.025261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:08.025261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:08.025261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:08.025261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:08.025261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:08.025261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:08.025261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:08.025261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:08.025261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:08.027180 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 66/125] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:08.028175 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182645af-f2de-4157-8216-f2d4879598f0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:08.049509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:08.057334 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac19bcc-3c2a-463e-bcae-a80e5cbc140f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:08.058096 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac19bcc-3c2a-463e-bcae-a80e5cbc140f None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:08.058489 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac19bcc-3c2a-463e-bcae-a80e5cbc140f None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:08.058876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac19bcc-3c2a-463e-bcae-a80e5cbc140f None admin] PATH_INFO: `/v3/users/87e4d594c1134b86837b4e4c464d9cfd` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:08.060920 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac19bcc-3c2a-463e-bcae-a80e5cbc140f None admin] RBAC: Authorizing `identity:get_user(user_id=87e4d594c1134b86837b4e4c464d9cfd)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:08.062511 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac19bcc-3c2a-463e-bcae-a80e5cbc140f None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:08.064243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 60/126] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:07 2022] GET /identity/v3/users/87e4d594c1134b86837b4e4c464d9cfd => generated 248 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:08.065195 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac19bcc-3c2a-463e-bcae-a80e5cbc140f None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:08.160098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:08.160565 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182645af-f2de-4157-8216-f2d4879598f0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:08.176075 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac19bcc-3c2a-463e-bcae-a80e5cbc140f None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:08.204377 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfb975-cd19-4a5d-89ba-1f440926492f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:08.205152 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfb975-cd19-4a5d-89ba-1f440926492f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:08.205530 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfb975-cd19-4a5d-89ba-1f440926492f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:08.205888 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfb975-cd19-4a5d-89ba-1f440926492f None admin] PATH_INFO: `/v3/users/b44229d01ea54d08a04c913b5f878f19` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:08.207168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfb975-cd19-4a5d-89ba-1f440926492f None admin] RBAC: Authorizing `identity:get_user(user_id=b44229d01ea54d08a04c913b5f878f19)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:08.208697 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfb975-cd19-4a5d-89ba-1f440926492f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:08.210426 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 67/127] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:08 2022] GET /identity/v3/users/b44229d01ea54d08a04c913b5f878f19 => generated 276 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:08.211365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfb975-cd19-4a5d-89ba-1f440926492f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:08.214516 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30c407-d31b-4565-af1b-b5d469664a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:08.215354 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30c407-d31b-4565-af1b-b5d469664a27 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:08.215649 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30c407-d31b-4565-af1b-b5d469664a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:08.216015 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30c407-d31b-4565-af1b-b5d469664a27 None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:08.217603 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30c407-d31b-4565-af1b-b5d469664a27 None admin] RBAC: Authorizing `identity:get_project(project_id=a78316c83e17485ebfd6d3c10ec19d8e)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:08.219070 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30c407-d31b-4565-af1b-b5d469664a27 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:08.221019 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 61/128] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:08 2022] GET /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e => generated 299 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:08.226404 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30c407-d31b-4565-af1b-b5d469664a27 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:08.297234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:08.318825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfb975-cd19-4a5d-89ba-1f440926492f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:08.341014 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30c407-d31b-4565-af1b-b5d469664a27 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:08.354801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4b363-6bae-41ec-9f90-18fe52a8fe85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:08.355603 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4b363-6bae-41ec-9f90-18fe52a8fe85 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:08.356794 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4b363-6bae-41ec-9f90-18fe52a8fe85 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:08.357088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4b363-6bae-41ec-9f90-18fe52a8fe85 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:08.358165 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4b363-6bae-41ec-9f90-18fe52a8fe85 None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:08.358483 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4b363-6bae-41ec-9f90-18fe52a8fe85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:08.360344 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4b363-6bae-41ec-9f90-18fe52a8fe85 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:08.370990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 68/129] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:08 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:08.372448 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4b363-6bae-41ec-9f90-18fe52a8fe85 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:08.383854 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a14e7e-6a39-484d-af43-1f2f25da2679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:08.384515 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a14e7e-6a39-484d-af43-1f2f25da2679 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:08.384854 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a14e7e-6a39-484d-af43-1f2f25da2679 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:08.385298 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a14e7e-6a39-484d-af43-1f2f25da2679 None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:08.386606 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a14e7e-6a39-484d-af43-1f2f25da2679 None admin] RBAC: Authorizing `identity:get_project(project_id=a78316c83e17485ebfd6d3c10ec19d8e)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:08.388242 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a14e7e-6a39-484d-af43-1f2f25da2679 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:08.390304 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 62/130] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:08 2022] GET /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e => generated 299 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:08.391240 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a14e7e-6a39-484d-af43-1f2f25da2679 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:08.425090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:08.507555 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a14e7e-6a39-484d-af43-1f2f25da2679 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:08.515235 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4b363-6bae-41ec-9f90-18fe52a8fe85 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:08.547885 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac70f7-18be-406a-8b10-2f08d7d80a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:08.548747 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac70f7-18be-406a-8b10-2f08d7d80a42 None admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:08.549156 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac70f7-18be-406a-8b10-2f08d7d80a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:08.549547 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac70f7-18be-406a-8b10-2f08d7d80a42 None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e/users/b44229d01ea54d08a04c913b5f878f19/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:08.551691 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac70f7-18be-406a-8b10-2f08d7d80a42 None admin] RBAC: Authorizing `identity:create_grant(project_id=a78316c83e17485ebfd6d3c10ec19d8e, user_id=b44229d01ea54d08a04c913b5f878f19, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:08.553456 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac70f7-18be-406a-8b10-2f08d7d80a42 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:08.560755 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed370473-d969-48cc-a278-1f4f0b4dd6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:08.561690 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed370473-d969-48cc-a278-1f4f0b4dd6de None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:08.562048 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed370473-d969-48cc-a278-1f4f0b4dd6de None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:08.562348 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed370473-d969-48cc-a278-1f4f0b4dd6de None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:08.563648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed370473-d969-48cc-a278-1f4f0b4dd6de None admin] RBAC: Authorizing `identity:get_project(project_id=a78316c83e17485ebfd6d3c10ec19d8e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:08.565245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed370473-d969-48cc-a278-1f4f0b4dd6de None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:08.567253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 69/131] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:08 2022] GET /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e => generated 299 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:08.567833 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed370473-d969-48cc-a278-1f4f0b4dd6de None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:08.571726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 63/132] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:57:08 2022] PUT /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e/users/b44229d01ea54d08a04c913b5f878f19/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:57:08.573206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac70f7-18be-406a-8b10-2f08d7d80a42 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:08.583050 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:08.689419 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:08.691684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed370473-d969-48cc-a278-1f4f0b4dd6de None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:08.712602 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac70f7-18be-406a-8b10-2f08d7d80a42 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:08.740202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457cacc7-39b0-4692-85b0-256559fbd938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:08.741333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457cacc7-39b0-4692-85b0-256559fbd938 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:08.742016 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457cacc7-39b0-4692-85b0-256559fbd938 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:08.742504 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457cacc7-39b0-4692-85b0-256559fbd938 None admin] PATH_INFO: `/v3/users/b44229d01ea54d08a04c913b5f878f19` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:08.744034 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457cacc7-39b0-4692-85b0-256559fbd938 None admin] RBAC: Authorizing `identity:get_user(user_id=b44229d01ea54d08a04c913b5f878f19)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:08.748361 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457cacc7-39b0-4692-85b0-256559fbd938 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:08.751817 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 70/133] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:08 2022] GET /identity/v3/users/b44229d01ea54d08a04c913b5f878f19 => generated 276 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:08.754400 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457cacc7-39b0-4692-85b0-256559fbd938 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:08.791683 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123d75f-83df-4e4f-ab9f-ee08c716dc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:08.792387 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123d75f-83df-4e4f-ab9f-ee08c716dc8d None admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:08.792671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123d75f-83df-4e4f-ab9f-ee08c716dc8d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:08.792951 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123d75f-83df-4e4f-ab9f-ee08c716dc8d None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e/users/87e4d594c1134b86837b4e4c464d9cfd/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:08.795616 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f123d75f-83df-4e4f-ab9f-ee08c716dc8d None admin] RBAC: Authorizing `identity:create_grant(project_id=a78316c83e17485ebfd6d3c10ec19d8e, user_id=87e4d594c1134b86837b4e4c464d9cfd, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:08.797379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f123d75f-83df-4e4f-ab9f-ee08c716dc8d None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:08.816192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 64/134] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:57:08 2022] PUT /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e/users/87e4d594c1134b86837b4e4c464d9cfd/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:57:08.817936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123d75f-83df-4e4f-ab9f-ee08c716dc8d None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:08.866622 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:08.892522 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457cacc7-39b0-4692-85b0-256559fbd938 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:08.952032 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41363747-89ac-45da-bd8e-a6de586d3d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:08.952665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41363747-89ac-45da-bd8e-a6de586d3d16 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:08.952958 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41363747-89ac-45da-bd8e-a6de586d3d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:08.953357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41363747-89ac-45da-bd8e-a6de586d3d16 None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e/users/b44229d01ea54d08a04c913b5f878f19/roles/bb986e49c38c40bab629ff32371b9358` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:08.960536 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41363747-89ac-45da-bd8e-a6de586d3d16 None admin] RBAC: Authorizing `identity:create_grant(project_id=a78316c83e17485ebfd6d3c10ec19d8e, user_id=b44229d01ea54d08a04c913b5f878f19, role_id=bb986e49c38c40bab629ff32371b9358)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:08.962061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41363747-89ac-45da-bd8e-a6de586d3d16 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:08.993052 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 71/135] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:57:08 2022] PUT /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e/users/b44229d01ea54d08a04c913b5f878f19/roles/bb986e49c38c40bab629ff32371b9358 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:57:08.995456 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41061194-d3bf-40f8-8b40-e7a4dedc2235 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:08.995979 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41061194-d3bf-40f8-8b40-e7a4dedc2235 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:08.997256 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41061194-d3bf-40f8-8b40-e7a4dedc2235 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:09.003424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123d75f-83df-4e4f-ab9f-ee08c716dc8d None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:09.003830 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 72/136] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:08 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:09.006819 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb80b18-a280-43f8-9a8c-98d1601369ca None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:09.008419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb80b18-a280-43f8-9a8c-98d1601369ca None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:09.008936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb80b18-a280-43f8-9a8c-98d1601369ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:09.031846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1bb80b18-a280-43f8-9a8c-98d1601369ca None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:09.033601 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1bb80b18-a280-43f8-9a8c-98d1601369ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:09.075879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1bbd1b-1324-401c-8926-c8af07405223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:09.076907 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bbd1b-1324-401c-8926-c8af07405223 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:09.077259 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bbd1b-1324-401c-8926-c8af07405223 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:09.077554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bbd1b-1324-401c-8926-c8af07405223 None admin] PATH_INFO: `/v3/projects/dfeb4e06d7754aa3940fc70d5d408a30` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:09.078860 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1bbd1b-1324-401c-8926-c8af07405223 None admin] RBAC: Authorizing `identity:get_project(project_id=dfeb4e06d7754aa3940fc70d5d408a30)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:09.080451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1bbd1b-1324-401c-8926-c8af07405223 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:09.082479 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 65/137] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:08 2022] GET /identity/v3/projects/dfeb4e06d7754aa3940fc70d5d408a30 => generated 313 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:09.088585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1bbd1b-1324-401c-8926-c8af07405223 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:09.120209 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:09.127912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 73/138] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:09 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:57:09.155467 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb80b18-a280-43f8-9a8c-98d1601369ca None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:09.241700 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:09.266076 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1bbd1b-1324-401c-8926-c8af07405223 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:09.291514 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb80b18-a280-43f8-9a8c-98d1601369ca None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:09.333732 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a44e47-d147-46a9-a644-42e8eab055b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:09.334939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a44e47-d147-46a9-a644-42e8eab055b5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:09.335460 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a44e47-d147-46a9-a644-42e8eab055b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:09.336036 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a44e47-d147-46a9-a644-42e8eab055b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:09.337523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a44e47-d147-46a9-a644-42e8eab055b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:09.339250 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a44e47-d147-46a9-a644-42e8eab055b5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:09.342100 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de24b08-a26b-419a-94e7-29e904c5938a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:09.342602 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de24b08-a26b-419a-94e7-29e904c5938a None admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:09.342835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 74/139] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:09 2022] GET /identity/v3/domains/default => generated 200 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:09.342939 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de24b08-a26b-419a-94e7-29e904c5938a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:09.343950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de24b08-a26b-419a-94e7-29e904c5938a None admin] PATH_INFO: `/v3/projects/dfeb4e06d7754aa3940fc70d5d408a30/users/b44229d01ea54d08a04c913b5f878f19/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:09.345212 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de24b08-a26b-419a-94e7-29e904c5938a None admin] RBAC: Authorizing `identity:create_grant(project_id=dfeb4e06d7754aa3940fc70d5d408a30, user_id=b44229d01ea54d08a04c913b5f878f19, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:09.346778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de24b08-a26b-419a-94e7-29e904c5938a None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:09.348532 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a44e47-d147-46a9-a644-42e8eab055b5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:09.374219 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 66/140] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:57:09 2022] PUT /identity/v3/projects/dfeb4e06d7754aa3940fc70d5d408a30/users/b44229d01ea54d08a04c913b5f878f19/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 08:57:09.477642 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a44e47-d147-46a9-a644-42e8eab055b5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:09.535632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaebe04-917b-440d-9475-3030b98e4c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:09.537392 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaebe04-917b-440d-9475-3030b98e4c03 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:09.537889 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaebe04-917b-440d-9475-3030b98e4c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:09.538360 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaebe04-917b-440d-9475-3030b98e4c03 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:09.539267 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaebe04-917b-440d-9475-3030b98e4c03 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:09.540935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaebe04-917b-440d-9475-3030b98e4c03 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:09.554421 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:09.582047 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 75/141] 10.222.0.100 () {60 vars in 1137 bytes} [Tue Aug 2 08:57:09 2022] POST /identity/v3/users => generated 284 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 08:57:10.070126 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec303da-b81d-4d0a-b9b2-7ca11d22f76d None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:10.070624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec303da-b81d-4d0a-b9b2-7ca11d22f76d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:10.070727 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec303da-b81d-4d0a-b9b2-7ca11d22f76d None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:10.072036 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 67/142] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:10.077361 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde1178-c68c-48ca-a859-8228ee0eba53 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:10.077659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde1178-c68c-48ca-a859-8228ee0eba53 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:10.077942 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde1178-c68c-48ca-a859-8228ee0eba53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:10.102474 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-fdde1178-c68c-48ca-a859-8228ee0eba53 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:10.103722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fdde1178-c68c-48ca-a859-8228ee0eba53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:10.183599 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 76/143] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:10 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:10.213278 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec303da-b81d-4d0a-b9b2-7ca11d22f76d None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:10.325683 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:10.352159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec303da-b81d-4d0a-b9b2-7ca11d22f76d None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:10.390994 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd9bad-7f86-4c4c-a2a4-ecbfa7197218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:10.391645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd9bad-7f86-4c4c-a2a4-ecbfa7197218 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:10.391934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd9bad-7f86-4c4c-a2a4-ecbfa7197218 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:10.392208 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd9bad-7f86-4c4c-a2a4-ecbfa7197218 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:10.403411 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd9bad-7f86-4c4c-a2a4-ecbfa7197218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:10.408347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-ffcd9bad-7f86-4c4c-a2a4-ecbfa7197218 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:57:10.408347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:10.408347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:10.408347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:10.408347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:10.408347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:10.408347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:10.408347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:10.408347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:10.408347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:10.408347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:10.408347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:10.408347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:10.408347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:10.408347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:10.408347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:10.409379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:10.409379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:10.409379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:10.409379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:10.409379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:10.409379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:10.409379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:10.409379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:10.409379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:10.409379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:10.409379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:10.409379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:10.409379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:10.409379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:10.409379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:10.409379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:10.410310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:10.410310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:10.410310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:10.410310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:10.410310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:10.410310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:10.410310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:10.410310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:10.410310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:10.410310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:10.410310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:57:10.410310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:10.410310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 68/144] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:10.415750 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde1178-c68c-48ca-a859-8228ee0eba53 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:10.536832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde1178-c68c-48ca-a859-8228ee0eba53 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:10.561419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:10.572854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6cca76-1862-425f-af94-77246e235999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:10.573620 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cca76-1862-425f-af94-77246e235999 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:10.573939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cca76-1862-425f-af94-77246e235999 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:10.574266 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cca76-1862-425f-af94-77246e235999 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:10.575310 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6cca76-1862-425f-af94-77246e235999 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:10.575609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6cca76-1862-425f-af94-77246e235999 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:10.577754 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6cca76-1862-425f-af94-77246e235999 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:10.586840 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 77/145] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:10 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:10.592573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd9bad-7f86-4c4c-a2a4-ecbfa7197218 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:10.700150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd9bad-7f86-4c4c-a2a4-ecbfa7197218 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:10.738071 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fd1c3-973c-4e6c-abe7-bfcc97c71bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:10.738727 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fd1c3-973c-4e6c-abe7-bfcc97c71bde None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:10.739010 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fd1c3-973c-4e6c-abe7-bfcc97c71bde None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:10.739291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fd1c3-973c-4e6c-abe7-bfcc97c71bde None admin] PATH_INFO: `/v3/users/a716df57e0834669a75c53f672b102ea` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:10.740469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fd1c3-973c-4e6c-abe7-bfcc97c71bde None admin] RBAC: Authorizing `identity:get_user(user_id=a716df57e0834669a75c53f672b102ea)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:10.741970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fd1c3-973c-4e6c-abe7-bfcc97c71bde None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:10.743818 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 69/146] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:10 2022] GET /identity/v3/users/a716df57e0834669a75c53f672b102ea => generated 290 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:10.749621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6cca76-1862-425f-af94-77246e235999 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:10.855076 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6cca76-1862-425f-af94-77246e235999 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:10.890715 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac7247-b2fe-434b-a059-c7f53e4e55e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:10.891372 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac7247-b2fe-434b-a059-c7f53e4e55e7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:10.891658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac7247-b2fe-434b-a059-c7f53e4e55e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:10.891936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac7247-b2fe-434b-a059-c7f53e4e55e7 None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:10.893132 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac7247-b2fe-434b-a059-c7f53e4e55e7 None admin] RBAC: Authorizing `identity:get_project(project_id=a78316c83e17485ebfd6d3c10ec19d8e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:10.894640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac7247-b2fe-434b-a059-c7f53e4e55e7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:10.896588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 78/147] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:10 2022] GET /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e => generated 299 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:10.902224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fd1c3-973c-4e6c-abe7-bfcc97c71bde None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:10.910281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:11.068024 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fd1c3-973c-4e6c-abe7-bfcc97c71bde None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:11.107777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca3f05-2ce1-40c9-bab4-beb980696f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:11.108486 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca3f05-2ce1-40c9-bab4-beb980696f61 None admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:11.108673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca3f05-2ce1-40c9-bab4-beb980696f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:11.108950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca3f05-2ce1-40c9-bab4-beb980696f61 None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e/users/a716df57e0834669a75c53f672b102ea/roles/8724fd98145549928aa27d32c24e610a` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:11.110960 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca3f05-2ce1-40c9-bab4-beb980696f61 None admin] RBAC: Authorizing `identity:create_grant(project_id=a78316c83e17485ebfd6d3c10ec19d8e, user_id=a716df57e0834669a75c53f672b102ea, role_id=8724fd98145549928aa27d32c24e610a)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:11.112428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca3f05-2ce1-40c9-bab4-beb980696f61 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:11.145871 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 70/148] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:57:10 2022] PUT /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e/users/a716df57e0834669a75c53f672b102ea/roles/8724fd98145549928aa27d32c24e610a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 08:57:11.957543 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98830e7d-97a7-4869-ae85-7bc1ed66baaf None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:11.958105 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98830e7d-97a7-4869-ae85-7bc1ed66baaf None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:11.958105 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98830e7d-97a7-4869-ae85-7bc1ed66baaf None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:11.959449 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 79/149] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:11.971594 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113026e-9853-45d1-abe1-74bf6a5490bc None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:11.971879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113026e-9853-45d1-abe1-74bf6a5490bc None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:11.972340 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113026e-9853-45d1-abe1-74bf6a5490bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:12.001266 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-2113026e-9853-45d1-abe1-74bf6a5490bc None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:12.002991 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-2113026e-9853-45d1-abe1-74bf6a5490bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:12.080776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:12.092407 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 71/150] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:11 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:12.127162 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98830e7d-97a7-4869-ae85-7bc1ed66baaf None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:12.245594 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:12.289769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98830e7d-97a7-4869-ae85-7bc1ed66baaf None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:12.326917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1d40a-c80f-4875-ac84-a1bd34ecc2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:12.327580 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1d40a-c80f-4875-ac84-a1bd34ecc2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:12.327869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1d40a-c80f-4875-ac84-a1bd34ecc2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:12.328146 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1d40a-c80f-4875-ac84-a1bd34ecc2e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:12.339533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d1d40a-c80f-4875-ac84-a1bd34ecc2e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:12.345401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-89d1d40a-c80f-4875-ac84-a1bd34ecc2e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:12.345401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:12.345401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:12.345401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:12.345401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:12.345401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:12.345401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:12.345401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:12.345401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:12.345401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:12.345401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:12.345401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:12.345401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:12.345401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:12.345401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:12.345401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:12.346325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:12.346325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:12.346325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:12.346325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:12.346325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:12.346325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:12.346325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:12.346325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:12.346325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:12.346325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:12.346325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:12.346325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:12.346325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:12.346325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:12.346325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:12.346325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:12.347555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:12.347555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:12.347555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:12.347555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:12.347555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:12.347555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:12.347555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:12.347555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:12.347555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:12.347555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:12.347555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:12.347555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:12.347555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 80/151] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:12.352835 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113026e-9853-45d1-abe1-74bf6a5490bc None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:12.484916 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113026e-9853-45d1-abe1-74bf6a5490bc None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:12.535354 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4fe33-ce7d-4325-908f-7a3769ee63b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:12.536076 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4fe33-ce7d-4325-908f-7a3769ee63b8 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:12.536364 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4fe33-ce7d-4325-908f-7a3769ee63b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:12.536643 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4fe33-ce7d-4325-908f-7a3769ee63b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:12.537408 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4fe33-ce7d-4325-908f-7a3769ee63b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:12.537796 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4fe33-ce7d-4325-908f-7a3769ee63b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:12.539198 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4fe33-ce7d-4325-908f-7a3769ee63b8 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:12.548374 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 72/152] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:12 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:12.554055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1d40a-c80f-4875-ac84-a1bd34ecc2e7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:12.599553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d632b-ccfd-4a51-aa78-208ebe1bdd06 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:12.600185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d632b-ccfd-4a51-aa78-208ebe1bdd06 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:12.600672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d632b-ccfd-4a51-aa78-208ebe1bdd06 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:12.602527 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 73/153] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:12 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:12.608309 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329386af-ba48-47dc-9456-14e398faf2a8 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:12.608869 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329386af-ba48-47dc-9456-14e398faf2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:12.609416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329386af-ba48-47dc-9456-14e398faf2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:12.628078 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-329386af-ba48-47dc-9456-14e398faf2a8 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:12.629673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-329386af-ba48-47dc-9456-14e398faf2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:12.681828 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1d40a-c80f-4875-ac84-a1bd34ecc2e7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:12.685467 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:12.696248 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:12.707747 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 74/154] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:12 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:57:12.709336 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ee6af-fe8f-456a-ad1c-57b622d360e9 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:12.709746 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ee6af-fe8f-456a-ad1c-57b622d360e9 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:12.710115 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ee6af-fe8f-456a-ad1c-57b622d360e9 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:12.711719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 75/155] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:12.717940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738afb8-53e4-4ee6-9a8e-678cd6a5cd04 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:12.718505 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738afb8-53e4-4ee6-9a8e-678cd6a5cd04 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:12.718992 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738afb8-53e4-4ee6-9a8e-678cd6a5cd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:12.721585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961276f-9d26-4d90-9906-144b49c0249a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:12.722508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961276f-9d26-4d90-9906-144b49c0249a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:12.722964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961276f-9d26-4d90-9906-144b49c0249a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:12.723397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961276f-9d26-4d90-9906-144b49c0249a None admin] PATH_INFO: `/v3/users/b44229d01ea54d08a04c913b5f878f19` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:12.724788 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961276f-9d26-4d90-9906-144b49c0249a None admin] RBAC: Authorizing `identity:get_user(user_id=b44229d01ea54d08a04c913b5f878f19)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:12.726457 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961276f-9d26-4d90-9906-144b49c0249a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:12.728536 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 81/156] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:12 2022] GET /identity/v3/users/b44229d01ea54d08a04c913b5f878f19 => generated 276 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:12.734195 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961276f-9d26-4d90-9906-144b49c0249a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:12.744525 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-1738afb8-53e4-4ee6-9a8e-678cd6a5cd04 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:12.745872 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1738afb8-53e4-4ee6-9a8e-678cd6a5cd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:12.841165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 76/157] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:12 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:12.842339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738afb8-53e4-4ee6-9a8e-678cd6a5cd04 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:12.842461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961276f-9d26-4d90-9906-144b49c0249a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:12.879020 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e791c-c804-4e93-b031-22dd13914484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:12.879662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e791c-c804-4e93-b031-22dd13914484 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:12.879949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e791c-c804-4e93-b031-22dd13914484 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:12.880230 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e791c-c804-4e93-b031-22dd13914484 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:12.931586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:12.952992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e791c-c804-4e93-b031-22dd13914484 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:12.954650 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e791c-c804-4e93-b031-22dd13914484 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:12.959781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738afb8-53e4-4ee6-9a8e-678cd6a5cd04 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:12.982573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:13.009473 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce908ad-2d99-4d83-84b7-1312f95bf6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:13.010463 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce908ad-2d99-4d83-84b7-1312f95bf6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:13.010958 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce908ad-2d99-4d83-84b7-1312f95bf6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:13.011692 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce908ad-2d99-4d83-84b7-1312f95bf6b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:13.031074 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce908ad-2d99-4d83-84b7-1312f95bf6b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:13.041137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-8ce908ad-2d99-4d83-84b7-1312f95bf6b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:13.041137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:13.041137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:13.041137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:13.041137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:13.041137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:13.041137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:13.041137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:13.041137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:13.041137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:13.041137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:13.041137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:13.041137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:13.041137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:13.041137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:13.041137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:13.042118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:13.042118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:13.042118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:13.042118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:13.042118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:13.042118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:13.042118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:13.042118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:13.042118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:13.042118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:13.042118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:13.042118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:13.042118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:13.042118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:13.042118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:13.042118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:13.042884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:13.042884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:13.042884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:13.042884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:13.042884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:13.042884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:13.042884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:13.042884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:13.042884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:13.042884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:13.042884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:13.042884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:13.046140 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 77/158] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:57:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:13.047182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce908ad-2d99-4d83-84b7-1312f95bf6b3 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:13.060360 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 82/159] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:12 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:13.065352 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e791c-c804-4e93-b031-22dd13914484 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:13.149570 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:13.170190 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e791c-c804-4e93-b031-22dd13914484 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:13.205988 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce908ad-2d99-4d83-84b7-1312f95bf6b3 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:13.207794 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd8ff7-5617-4e5d-8fb7-47ce2bdddf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:13.208694 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd8ff7-5617-4e5d-8fb7-47ce2bdddf72 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:13.209221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd8ff7-5617-4e5d-8fb7-47ce2bdddf72 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:13.209697 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd8ff7-5617-4e5d-8fb7-47ce2bdddf72 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:13.210664 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd8ff7-5617-4e5d-8fb7-47ce2bdddf72 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:13.211172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd8ff7-5617-4e5d-8fb7-47ce2bdddf72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:13.212746 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd8ff7-5617-4e5d-8fb7-47ce2bdddf72 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:13.222787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 83/160] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:57:13 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:13.223962 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd8ff7-5617-4e5d-8fb7-47ce2bdddf72 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:13.269819 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56f2dc-5a20-4206-825d-e1ecbff2b920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:13.270753 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56f2dc-5a20-4206-825d-e1ecbff2b920 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:13.271264 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56f2dc-5a20-4206-825d-e1ecbff2b920 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:13.271743 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56f2dc-5a20-4206-825d-e1ecbff2b920 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:13.286099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56f2dc-5a20-4206-825d-e1ecbff2b920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:13.296175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-eb56f2dc-5a20-4206-825d-e1ecbff2b920 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:57:13.296175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:13.296175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:13.296175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:13.296175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:13.296175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:13.296175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:13.296175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:13.296175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:13.296175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:13.296175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:13.296175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:13.296175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:13.296175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:13.296175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:13.296175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:13.297159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:13.297159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:13.297159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:13.297159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:13.297159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:13.297159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:13.297159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:13.297159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:13.297159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:13.297159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:13.297159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:13.297159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:13.297159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:13.297159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:13.297159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:13.297159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:13.297940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:13.297940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:13.297940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:13.297940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:13.297940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:13.297940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:13.297940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:13.297940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:13.297940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:13.297940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:13.297940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:57:13.297940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:13.300290 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 78/161] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:57:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:13.301294 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56f2dc-5a20-4206-825d-e1ecbff2b920 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:13.339199 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd8ff7-5617-4e5d-8fb7-47ce2bdddf72 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:13.361557 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:13.376341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79b8f8-bd9c-4e1c-91ca-372c3eec3689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:13.377633 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79b8f8-bd9c-4e1c-91ca-372c3eec3689 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:13.378124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79b8f8-bd9c-4e1c-91ca-372c3eec3689 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:13.378613 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79b8f8-bd9c-4e1c-91ca-372c3eec3689 None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:13.380020 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79b8f8-bd9c-4e1c-91ca-372c3eec3689 None admin] RBAC: Authorizing `identity:get_project(project_id=a78316c83e17485ebfd6d3c10ec19d8e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:13.381730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79b8f8-bd9c-4e1c-91ca-372c3eec3689 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:13.383660 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 84/162] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:13 2022] GET /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e => generated 299 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:13.384621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79b8f8-bd9c-4e1c-91ca-372c3eec3689 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:13.413588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:13.469625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56f2dc-5a20-4206-825d-e1ecbff2b920 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:13.524681 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea703b38-0c86-4329-af44-033d22f3fb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:13.525773 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea703b38-0c86-4329-af44-033d22f3fb8e None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:13.527115 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea703b38-0c86-4329-af44-033d22f3fb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:13.527115 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea703b38-0c86-4329-af44-033d22f3fb8e None admin] PATH_INFO: `/v3/users/b44229d01ea54d08a04c913b5f878f19` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:13.530222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea703b38-0c86-4329-af44-033d22f3fb8e None admin] RBAC: Authorizing `identity:get_user(user_id=b44229d01ea54d08a04c913b5f878f19)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:13.531704 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea703b38-0c86-4329-af44-033d22f3fb8e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:13.533403 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 79/163] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:13 2022] GET /identity/v3/users/b44229d01ea54d08a04c913b5f878f19 => generated 276 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:13.534176 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea703b38-0c86-4329-af44-033d22f3fb8e None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:13.669326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79b8f8-bd9c-4e1c-91ca-372c3eec3689 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:13.681041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea703b38-0c86-4329-af44-033d22f3fb8e None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:13.686859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:13.742434 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b799296-41cf-4b37-a421-d54197351516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:13.743081 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b799296-41cf-4b37-a421-d54197351516 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:13.743373 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b799296-41cf-4b37-a421-d54197351516 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:13.743654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b799296-41cf-4b37-a421-d54197351516 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:13.744392 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b799296-41cf-4b37-a421-d54197351516 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:13.744710 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b799296-41cf-4b37-a421-d54197351516 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:13.746658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b799296-41cf-4b37-a421-d54197351516 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:13.748657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b756b-5326-479b-9ef6-5dc34190c6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:13.749409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b756b-5326-479b-9ef6-5dc34190c6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:13.749693 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b756b-5326-479b-9ef6-5dc34190c6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:13.749967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b756b-5326-479b-9ef6-5dc34190c6a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:13.750758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b756b-5326-479b-9ef6-5dc34190c6a8 None admin] RBAC: Adding query filter params (role.id=f739a9c857cd4050896a6814c8bc0120, scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e, user.id=b44229d01ea54d08a04c913b5f878f19) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:13.751073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b756b-5326-479b-9ef6-5dc34190c6a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:13.752873 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b756b-5326-479b-9ef6-5dc34190c6a8 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:13.760770 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 85/164] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:13 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:13.761644 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b799296-41cf-4b37-a421-d54197351516 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:13.774066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 80/165] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:13 2022] GET /identity/v3/role_assignments?user.id=b44229d01ea54d08a04c913b5f878f19&scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e&role.id=f739a9c857cd4050896a6814c8bc0120 => generated 610 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:13.778690 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95ae62-406d-4669-a7b6-25950d224c0b None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:13.778977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95ae62-406d-4669-a7b6-25950d224c0b None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:13.779256 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95ae62-406d-4669-a7b6-25950d224c0b None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:13.781522 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 81/166] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:13 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:13.782683 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95ae62-406d-4669-a7b6-25950d224c0b None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:13.988100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:14.018863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95ae62-406d-4669-a7b6-25950d224c0b None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:14.048342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b799296-41cf-4b37-a421-d54197351516 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:14.094822 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90adcc-f232-4395-8960-ee93624e960e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:14.095489 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90adcc-f232-4395-8960-ee93624e960e None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:14.095770 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90adcc-f232-4395-8960-ee93624e960e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:14.096044 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90adcc-f232-4395-8960-ee93624e960e None admin] PATH_INFO: `/v3/users/87e4d594c1134b86837b4e4c464d9cfd` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:14.097265 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c90adcc-f232-4395-8960-ee93624e960e None admin] RBAC: Authorizing `identity:get_user(user_id=87e4d594c1134b86837b4e4c464d9cfd)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:14.098751 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c90adcc-f232-4395-8960-ee93624e960e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:14.100412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 82/167] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:13 2022] GET /identity/v3/users/87e4d594c1134b86837b4e4c464d9cfd => generated 248 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:14.101891 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671540f8-0c84-45be-89e0-73ebf9362117 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:14.101891 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671540f8-0c84-45be-89e0-73ebf9362117 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:14.102185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671540f8-0c84-45be-89e0-73ebf9362117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:14.134625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-671540f8-0c84-45be-89e0-73ebf9362117 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:14.135915 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-671540f8-0c84-45be-89e0-73ebf9362117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:14.147249 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4498561-106a-4b35-8ae5-83889565dd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:14.147896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4498561-106a-4b35-8ae5-83889565dd64 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:14.148181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4498561-106a-4b35-8ae5-83889565dd64 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:14.148458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4498561-106a-4b35-8ae5-83889565dd64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:14.198935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:14.248972 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4498561-106a-4b35-8ae5-83889565dd64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:14.250633 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4498561-106a-4b35-8ae5-83889565dd64 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:14.303399 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 83/168] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:14.304275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671540f8-0c84-45be-89e0-73ebf9362117 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:14.349941 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:14.393215 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 86/169] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:13 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:14.395369 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646abcc-f013-4f26-842c-218583cc57dc None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:14.395659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646abcc-f013-4f26-842c-218583cc57dc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:14.395942 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646abcc-f013-4f26-842c-218583cc57dc None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:14.401797 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 87/170] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:14 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:14.403082 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646abcc-f013-4f26-842c-218583cc57dc None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:14.542746 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671540f8-0c84-45be-89e0-73ebf9362117 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:14.560520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646abcc-f013-4f26-842c-218583cc57dc None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:14.609288 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7a6c2-87bc-422c-89bd-93c29c1de7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:14.610365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7a6c2-87bc-422c-89bd-93c29c1de7a7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:14.610830 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c131d-8e31-45a0-bb3c-c300b2027f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:14.611030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7a6c2-87bc-422c-89bd-93c29c1de7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:14.611479 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7a6c2-87bc-422c-89bd-93c29c1de7a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:14.611577 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c131d-8e31-45a0-bb3c-c300b2027f26 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:14.611764 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c131d-8e31-45a0-bb3c-c300b2027f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:14.612041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c131d-8e31-45a0-bb3c-c300b2027f26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:14.625363 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7a6c2-87bc-422c-89bd-93c29c1de7a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:14.631314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-05c7a6c2-87bc-422c-89bd-93c29c1de7a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:14.631314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:14.631314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:14.631314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:14.631314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:14.631314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:14.631314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:14.631314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:14.631314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:14.631314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:14.631314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:14.631314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:14.631314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:14.631314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:14.631314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:14.631314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:14.632189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:14.632189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:14.632189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:14.632189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:14.632189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:14.632189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:14.632189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:14.632189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:14.632189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:14.632189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:14.632189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:14.632189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:14.632189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:14.632189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:14.632189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:14.632189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:14.632990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:14.632990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:14.632990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:14.632990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:14.632990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:14.632990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:14.632990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:14.632990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:14.632990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:14.632990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:14.632990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:14.632990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:14.635515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 88/171] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:14.636578 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7a6c2-87bc-422c-89bd-93c29c1de7a7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:14.682765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:14.695608 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:14.748660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c131d-8e31-45a0-bb3c-c300b2027f26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:14.750995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c131d-8e31-45a0-bb3c-c300b2027f26 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:14.776444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7a6c2-87bc-422c-89bd-93c29c1de7a7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:14.814596 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f9304-9311-4a25-b682-1b70e2bc1ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:14.815156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f9304-9311-4a25-b682-1b70e2bc1ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:14.815443 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f9304-9311-4a25-b682-1b70e2bc1ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:14.815720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f9304-9311-4a25-b682-1b70e2bc1ea3 None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:14.817147 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f9304-9311-4a25-b682-1b70e2bc1ea3 None admin] RBAC: Authorizing `identity:get_project(project_id=a78316c83e17485ebfd6d3c10ec19d8e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:14.818406 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f9304-9311-4a25-b682-1b70e2bc1ea3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:14.825017 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 89/172] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:14 2022] GET /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e => generated 299 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:14.825017 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f027dd-70e6-48e5-a167-a94da9983e86 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:14.825017 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f027dd-70e6-48e5-a167-a94da9983e86 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:14.825017 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f027dd-70e6-48e5-a167-a94da9983e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:14.843054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-00f027dd-70e6-48e5-a167-a94da9983e86 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:14.844257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-00f027dd-70e6-48e5-a167-a94da9983e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:14.874177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 84/173] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:14 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:14.875718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c131d-8e31-45a0-bb3c-c300b2027f26 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:14.925134 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 90/174] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:14.925134 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a93dc-fbba-4f8f-be62-67c4e6438a97 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:14.925134 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a93dc-fbba-4f8f-be62-67c4e6438a97 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:14.925134 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a93dc-fbba-4f8f-be62-67c4e6438a97 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:14.925134 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 91/175] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:14.925134 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db47cc7-0dc8-4f08-a972-3be5237c4678 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:14.925134 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db47cc7-0dc8-4f08-a972-3be5237c4678 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:14.926154 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db47cc7-0dc8-4f08-a972-3be5237c4678 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:14.928541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 92/176] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:14 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:14.929333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db47cc7-0dc8-4f08-a972-3be5237c4678 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:14.958022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:14.964909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:14.995935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c131d-8e31-45a0-bb3c-c300b2027f26 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:15.030775 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c297a-014d-4616-bd3a-ffbfa075f49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:15.031393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c297a-014d-4616-bd3a-ffbfa075f49d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:15.031677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c297a-014d-4616-bd3a-ffbfa075f49d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:15.031951 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c297a-014d-4616-bd3a-ffbfa075f49d None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:15.032656 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c297a-014d-4616-bd3a-ffbfa075f49d None admin] RBAC: Adding query filter params (name=member) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:15.032961 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c297a-014d-4616-bd3a-ffbfa075f49d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:15.034372 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c297a-014d-4616-bd3a-ffbfa075f49d None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:15.040860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db47cc7-0dc8-4f08-a972-3be5237c4678 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:15.042241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 85/177] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:14 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:15.043110 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c297a-014d-4616-bd3a-ffbfa075f49d None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:15.076262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6d0ef-6883-400b-95d0-15d8cb8fc868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:15.076806 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6d0ef-6883-400b-95d0-15d8cb8fc868 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:15.077097 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6d0ef-6883-400b-95d0-15d8cb8fc868 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:15.077400 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6d0ef-6883-400b-95d0-15d8cb8fc868 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:15.078202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6d0ef-6883-400b-95d0-15d8cb8fc868 None admin] RBAC: Adding query filter params (role.id=bb986e49c38c40bab629ff32371b9358, scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e, user.id=b44229d01ea54d08a04c913b5f878f19) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:15.078516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6d0ef-6883-400b-95d0-15d8cb8fc868 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:15.079956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6d0ef-6883-400b-95d0-15d8cb8fc868 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:15.085751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 93/178] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:14 2022] GET /identity/v3/role_assignments?user.id=b44229d01ea54d08a04c913b5f878f19&scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e&role.id=bb986e49c38c40bab629ff32371b9358 => generated 610 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:15.086960 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8014c-fb00-4094-b0ce-6a5ecaa8e2f5 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:15.087257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8014c-fb00-4094-b0ce-6a5ecaa8e2f5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:15.087535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8014c-fb00-4094-b0ce-6a5ecaa8e2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:15.114234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1eb8014c-fb00-4094-b0ce-6a5ecaa8e2f5 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:15.115478 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1eb8014c-fb00-4094-b0ce-6a5ecaa8e2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:15.151977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c297a-014d-4616-bd3a-ffbfa075f49d None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:15.185816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:15.193825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 94/179] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:15.197685 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061acf0e-94c8-4f70-85ac-5a51873fcffa None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:15.197685 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061acf0e-94c8-4f70-85ac-5a51873fcffa None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:15.197685 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061acf0e-94c8-4f70-85ac-5a51873fcffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:15.202632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf8e2e-e7d6-49bf-b16d-f45e5e5f3928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:15.204855 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf8e2e-e7d6-49bf-b16d-f45e5e5f3928 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:15.205424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf8e2e-e7d6-49bf-b16d-f45e5e5f3928 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:15.205918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf8e2e-e7d6-49bf-b16d-f45e5e5f3928 None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:15.207447 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cf8e2e-e7d6-49bf-b16d-f45e5e5f3928 None admin] RBAC: Authorizing `identity:get_project(project_id=a78316c83e17485ebfd6d3c10ec19d8e)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:15.209203 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cf8e2e-e7d6-49bf-b16d-f45e5e5f3928 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:15.212129 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 86/180] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:15 2022] GET /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e => generated 299 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:15.213264 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf8e2e-e7d6-49bf-b16d-f45e5e5f3928 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:15.216652 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-061acf0e-94c8-4f70-85ac-5a51873fcffa None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:15.218693 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-061acf0e-94c8-4f70-85ac-5a51873fcffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:15.241143 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:15.300178 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 95/181] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:15.301657 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2e8fe-15a2-4acf-8eb5-2585d50fc88e None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:15.301952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2e8fe-15a2-4acf-8eb5-2585d50fc88e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:15.302231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2e8fe-15a2-4acf-8eb5-2585d50fc88e None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:15.303514 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 96/182] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:15.304272 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2e8fe-15a2-4acf-8eb5-2585d50fc88e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:15.341933 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf8e2e-e7d6-49bf-b16d-f45e5e5f3928 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:15.374998 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14539c6f-291c-44c3-833b-60845f2d2a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:15.375642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14539c6f-291c-44c3-833b-60845f2d2a0e None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:15.375918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14539c6f-291c-44c3-833b-60845f2d2a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:15.376190 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14539c6f-291c-44c3-833b-60845f2d2a0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:15.386311 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14539c6f-291c-44c3-833b-60845f2d2a0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:15.392273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-14539c6f-291c-44c3-833b-60845f2d2a0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:57:15.392273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:15.392273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:15.392273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:15.392273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:15.392273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:15.392273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:15.392273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:15.392273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:15.392273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:15.392273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:15.392273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:15.392273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:15.392273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:15.392273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:15.392273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:15.393181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:15.393181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:15.393181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:15.393181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:15.393181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:15.393181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:15.393181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:15.393181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:15.393181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:15.393181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:15.393181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:15.393181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:15.393181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:15.393181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:15.393181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:15.393181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:15.393966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:15.393966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:15.393966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:15.393966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:15.393966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:15.393966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:15.393966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:15.393966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:15.393966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:15.393966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:15.393966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:57:15.393966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:15.393966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 87/183] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:57:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:15.394607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14539c6f-291c-44c3-833b-60845f2d2a0e None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:15.417056 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2e8fe-15a2-4acf-8eb5-2585d50fc88e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:15.450069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:15.452440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e8da2-ce4c-4b4f-a858-082c48552ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:15.453147 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e8da2-ce4c-4b4f-a858-082c48552ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:15.453427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e8da2-ce4c-4b4f-a858-082c48552ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:15.453710 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e8da2-ce4c-4b4f-a858-082c48552ee4 None admin] PATH_INFO: `/v3/users/b44229d01ea54d08a04c913b5f878f19` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:15.454898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e8da2-ce4c-4b4f-a858-082c48552ee4 None admin] RBAC: Authorizing `identity:get_user(user_id=b44229d01ea54d08a04c913b5f878f19)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:15.456328 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e8da2-ce4c-4b4f-a858-082c48552ee4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:15.457969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 97/184] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:15 2022] GET /identity/v3/users/b44229d01ea54d08a04c913b5f878f19 => generated 276 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:15.462622 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e8da2-ce4c-4b4f-a858-082c48552ee4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:15.482673 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:15.497233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14539c6f-291c-44c3-833b-60845f2d2a0e None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:15.530748 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03a603-6482-4493-915d-9844db70badf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:15.531364 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03a603-6482-4493-915d-9844db70badf None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:15.531648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03a603-6482-4493-915d-9844db70badf None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:15.531922 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03a603-6482-4493-915d-9844db70badf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:15.532706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03a603-6482-4493-915d-9844db70badf None admin] RBAC: Adding query filter params (role.id=7cbff51867a7475d837afd51f6c12f11, scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e, user.id=87e4d594c1134b86837b4e4c464d9cfd) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:15.533016 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03a603-6482-4493-915d-9844db70badf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:15.534548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03a603-6482-4493-915d-9844db70badf None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:15.540239 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 88/185] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:15 2022] GET /identity/v3/role_assignments?user.id=87e4d594c1134b86837b4e4c464d9cfd&scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e&role.id=7cbff51867a7475d837afd51f6c12f11 => generated 610 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:15.540239 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07ac54-e3ba-4fcb-9da4-b2d8ed5de53b None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:15.540239 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07ac54-e3ba-4fcb-9da4-b2d8ed5de53b None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:15.540239 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07ac54-e3ba-4fcb-9da4-b2d8ed5de53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:15.567889 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e8da2-ce4c-4b4f-a858-082c48552ee4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:15.574183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-cd07ac54-e3ba-4fcb-9da4-b2d8ed5de53b None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:15.576058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-cd07ac54-e3ba-4fcb-9da4-b2d8ed5de53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:15.608449 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1236e-2db1-4b47-b16d-af214903148f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:15.609036 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1236e-2db1-4b47-b16d-af214903148f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:15.609356 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1236e-2db1-4b47-b16d-af214903148f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:15.609646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1236e-2db1-4b47-b16d-af214903148f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:15.621148 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e1236e-2db1-4b47-b16d-af214903148f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:15.626487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-b8e1236e-2db1-4b47-b16d-af214903148f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:57:15.626487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:15.626487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:15.626487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:15.626487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:15.626487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:15.626487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:15.626487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:15.626487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:15.626487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:15.626487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:15.626487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:15.626487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:15.626487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:15.626487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:15.626487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:15.627879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:15.627879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:15.627879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:15.627879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:15.627879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:15.627879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:15.627879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:15.627879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:15.627879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:15.627879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:15.627879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:15.627879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:15.627879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:15.627879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:15.627879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:15.627879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:15.628799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:15.628799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:15.628799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:15.628799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:15.628799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:15.628799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:15.628799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:15.628799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:15.628799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:15.628799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:15.628799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:57:15.628799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:15.628799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 98/186] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:15.628799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1236e-2db1-4b47-b16d-af214903148f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:15.658862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 89/187] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:15.660090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07ac54-e3ba-4fcb-9da4-b2d8ed5de53b None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:15.733385 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:15.733571 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:15.760786 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1236e-2db1-4b47-b16d-af214903148f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:15.788118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07ac54-e3ba-4fcb-9da4-b2d8ed5de53b None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:15.799688 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf2df2-ab48-4f8e-b88a-d5f35dae6b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:15.800335 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf2df2-ab48-4f8e-b88a-d5f35dae6b11 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:15.800618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf2df2-ab48-4f8e-b88a-d5f35dae6b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:15.800894 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf2df2-ab48-4f8e-b88a-d5f35dae6b11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:15.811527 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf2df2-ab48-4f8e-b88a-d5f35dae6b11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:15.816507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-08cf2df2-ab48-4f8e-b88a-d5f35dae6b11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:15.816507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:15.816507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:15.816507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:15.816507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:15.816507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:15.816507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:15.816507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:15.816507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:15.816507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:15.816507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:15.816507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:15.816507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:15.816507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:15.816507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:15.816507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:15.817528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:15.817528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:15.817528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:15.817528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:15.817528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:15.817528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:15.817528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:15.817528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:15.817528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:15.817528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:15.817528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:15.817528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:15.817528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:15.817528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:15.817528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:15.817528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:15.818500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:15.818500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:15.818500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:15.818500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:15.818500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:15.818500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:15.818500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:15.818500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:15.818500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:15.818500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:15.818500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:15.818500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:15.821425 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 99/188] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:57:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:15.822196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf2df2-ab48-4f8e-b88a-d5f35dae6b11 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:15.823639 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe1aee-2cb6-43a7-9afc-6bfa63143225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:15.824287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe1aee-2cb6-43a7-9afc-6bfa63143225 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:15.824745 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe1aee-2cb6-43a7-9afc-6bfa63143225 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:15.825020 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe1aee-2cb6-43a7-9afc-6bfa63143225 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:15.825767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe1aee-2cb6-43a7-9afc-6bfa63143225 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:15.826073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe1aee-2cb6-43a7-9afc-6bfa63143225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:15.827444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe1aee-2cb6-43a7-9afc-6bfa63143225 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:15.835643 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 90/189] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:15 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:15.838078 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe1aee-2cb6-43a7-9afc-6bfa63143225 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:15.923662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf2df2-ab48-4f8e-b88a-d5f35dae6b11 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:15.938033 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe1aee-2cb6-43a7-9afc-6bfa63143225 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:15.950681 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:15.959083 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2db0f-bae5-45ef-8e39-e5b85b16410c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:15.959715 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2db0f-bae5-45ef-8e39-e5b85b16410c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:15.960000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2db0f-bae5-45ef-8e39-e5b85b16410c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:15.960275 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2db0f-bae5-45ef-8e39-e5b85b16410c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:15.971608 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c8032-6179-41d2-a58d-161080a8b794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:15.972228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c8032-6179-41d2-a58d-161080a8b794 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:15.972518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c8032-6179-41d2-a58d-161080a8b794 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:15.972791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c8032-6179-41d2-a58d-161080a8b794 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:15.972909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:15.973543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c8032-6179-41d2-a58d-161080a8b794 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:15.973852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c8032-6179-41d2-a58d-161080a8b794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:15.975289 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c8032-6179-41d2-a58d-161080a8b794 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:15.984005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 91/190] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:15 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:15.984822 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c8032-6179-41d2-a58d-161080a8b794 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:16.031058 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2db0f-bae5-45ef-8e39-e5b85b16410c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:16.032599 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2db0f-bae5-45ef-8e39-e5b85b16410c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:16.083035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c8032-6179-41d2-a58d-161080a8b794 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:16.116819 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d53fa1-5530-4f57-bd25-861bd042880e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:16.117487 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d53fa1-5530-4f57-bd25-861bd042880e None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:16.117770 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d53fa1-5530-4f57-bd25-861bd042880e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:16.118041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d53fa1-5530-4f57-bd25-861bd042880e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:16.119436 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d53fa1-5530-4f57-bd25-861bd042880e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:16.120902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d53fa1-5530-4f57-bd25-861bd042880e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:16.122491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 92/191] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:15 2022] GET /identity/v3/domains/default => generated 200 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:16.123253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d53fa1-5530-4f57-bd25-861bd042880e None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:16.138993 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 100/192] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:15 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:16.138993 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2db0f-bae5-45ef-8e39-e5b85b16410c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:16.171127 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:16.178402 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:16.226048 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d53fa1-5530-4f57-bd25-861bd042880e None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:16.234735 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2db0f-bae5-45ef-8e39-e5b85b16410c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:16.259541 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73165c4-e90e-4b32-b2a7-fdf43cc4c048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:16.260164 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73165c4-e90e-4b32-b2a7-fdf43cc4c048 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:16.260447 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73165c4-e90e-4b32-b2a7-fdf43cc4c048 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:16.260720 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73165c4-e90e-4b32-b2a7-fdf43cc4c048 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:16.261466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73165c4-e90e-4b32-b2a7-fdf43cc4c048 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:16.261800 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73165c4-e90e-4b32-b2a7-fdf43cc4c048 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:16.263180 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73165c4-e90e-4b32-b2a7-fdf43cc4c048 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:16.268569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4baa6-5366-4e45-b932-97846629fa60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:16.269253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4baa6-5366-4e45-b932-97846629fa60 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:16.269539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4baa6-5366-4e45-b932-97846629fa60 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:16.269819 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4baa6-5366-4e45-b932-97846629fa60 None admin] PATH_INFO: `/v3/users/d5e07b8dbf3645eba64bf87ce3b1eb4c` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:16.270906 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 93/193] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:57:16 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:16.276221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73165c4-e90e-4b32-b2a7-fdf43cc4c048 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:16.283723 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4baa6-5366-4e45-b932-97846629fa60 None admin] RBAC: Authorizing `identity:get_user(user_id=d5e07b8dbf3645eba64bf87ce3b1eb4c)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:16.285196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4baa6-5366-4e45-b932-97846629fa60 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:16.286874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 101/194] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:16 2022] GET /identity/v3/users/d5e07b8dbf3645eba64bf87ce3b1eb4c => generated 284 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:16.289958 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4baa6-5366-4e45-b932-97846629fa60 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:16.375922 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73165c4-e90e-4b32-b2a7-fdf43cc4c048 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:16.378479 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:16.386032 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:16.390502 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4baa6-5366-4e45-b932-97846629fa60 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:16.412388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e976fa6-c057-46b5-b864-464538891155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:16.413027 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e976fa6-c057-46b5-b864-464538891155 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:16.413360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e976fa6-c057-46b5-b864-464538891155 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:16.413641 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e976fa6-c057-46b5-b864-464538891155 None admin] PATH_INFO: `/v3/users/a716df57e0834669a75c53f672b102ea` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:16.414824 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e976fa6-c057-46b5-b864-464538891155 None admin] RBAC: Authorizing `identity:get_user(user_id=a716df57e0834669a75c53f672b102ea)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:16.416270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e976fa6-c057-46b5-b864-464538891155 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:16.418209 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 94/195] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:16 2022] GET /identity/v3/users/a716df57e0834669a75c53f672b102ea => generated 290 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:16.418982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e976fa6-c057-46b5-b864-464538891155 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:16.425297 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff34a48-d6d6-4191-9b7a-cb15a03ed0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:16.425858 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff34a48-d6d6-4191-9b7a-cb15a03ed0c9 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:16.426144 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff34a48-d6d6-4191-9b7a-cb15a03ed0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:16.426418 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff34a48-d6d6-4191-9b7a-cb15a03ed0c9 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:16.427100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff34a48-d6d6-4191-9b7a-cb15a03ed0c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:16.428522 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff34a48-d6d6-4191-9b7a-cb15a03ed0c9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:16.451198 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 102/196] 10.222.0.100 () {60 vars in 1137 bytes} [Tue Aug 2 08:57:16 2022] POST /identity/v3/users => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:57:16.452372 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff34a48-d6d6-4191-9b7a-cb15a03ed0c9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:16.550029 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e976fa6-c057-46b5-b864-464538891155 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:16.584205 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff34a48-d6d6-4191-9b7a-cb15a03ed0c9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:16.587893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f865a0-85d5-4755-bada-0654327b6af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:16.588549 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f865a0-85d5-4755-bada-0654327b6af1 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:16.588836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f865a0-85d5-4755-bada-0654327b6af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:16.589130 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f865a0-85d5-4755-bada-0654327b6af1 None admin] PATH_INFO: `/v3/projects/dfeb4e06d7754aa3940fc70d5d408a30` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:16.591713 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f865a0-85d5-4755-bada-0654327b6af1 None admin] RBAC: Authorizing `identity:get_project(project_id=dfeb4e06d7754aa3940fc70d5d408a30)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:16.593225 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f865a0-85d5-4755-bada-0654327b6af1 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:16.599869 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 95/197] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:16 2022] GET /identity/v3/projects/dfeb4e06d7754aa3940fc70d5d408a30 => generated 313 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:16.599869 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f865a0-85d5-4755-bada-0654327b6af1 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:16.632622 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163571a-9afa-4ac4-bd5c-8e6d9127a198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:16.633358 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163571a-9afa-4ac4-bd5c-8e6d9127a198 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:16.633648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163571a-9afa-4ac4-bd5c-8e6d9127a198 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:16.633935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163571a-9afa-4ac4-bd5c-8e6d9127a198 None admin] PATH_INFO: `/v3/users/d5e07b8dbf3645eba64bf87ce3b1eb4c` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:16.635126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163571a-9afa-4ac4-bd5c-8e6d9127a198 None admin] RBAC: Authorizing `identity:get_user(user_id=d5e07b8dbf3645eba64bf87ce3b1eb4c)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:16.637221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163571a-9afa-4ac4-bd5c-8e6d9127a198 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:16.638820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 103/198] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:16 2022] GET /identity/v3/users/d5e07b8dbf3645eba64bf87ce3b1eb4c => generated 284 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:16.639594 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163571a-9afa-4ac4-bd5c-8e6d9127a198 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:16.651691 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:16.670714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:16.725990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f865a0-85d5-4755-bada-0654327b6af1 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:16.771295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a126a0b-97ab-4246-b728-669e21bd4480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:16.771961 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a126a0b-97ab-4246-b728-669e21bd4480 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:16.772247 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a126a0b-97ab-4246-b728-669e21bd4480 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:16.772528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a126a0b-97ab-4246-b728-669e21bd4480 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:16.774814 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163571a-9afa-4ac4-bd5c-8e6d9127a198 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:16.819456 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250e1a4-b2f9-4ab5-ba4c-1d45fe902e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:16.820094 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250e1a4-b2f9-4ab5-ba4c-1d45fe902e42 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:16.820383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250e1a4-b2f9-4ab5-ba4c-1d45fe902e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:16.820664 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250e1a4-b2f9-4ab5-ba4c-1d45fe902e42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:16.864488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a126a0b-97ab-4246-b728-669e21bd4480 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:16.866338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a126a0b-97ab-4246-b728-669e21bd4480 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:16.897074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250e1a4-b2f9-4ab5-ba4c-1d45fe902e42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:16.898700 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250e1a4-b2f9-4ab5-ba4c-1d45fe902e42 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:16.920132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:16.957536 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:16.999701 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 96/199] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:16 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:17.000482 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a126a0b-97ab-4246-b728-669e21bd4480 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:17.033019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 104/200] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:16 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:17.034456 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250e1a4-b2f9-4ab5-ba4c-1d45fe902e42 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:17.118101 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a126a0b-97ab-4246-b728-669e21bd4480 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:17.156716 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e297a2-a408-4eb2-af73-d3f87231139f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:17.157301 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250e1a4-b2f9-4ab5-ba4c-1d45fe902e42 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:17.157963 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e297a2-a408-4eb2-af73-d3f87231139f None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:17.158453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e297a2-a408-4eb2-af73-d3f87231139f None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:17.158921 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e297a2-a408-4eb2-af73-d3f87231139f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:17.159950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e297a2-a408-4eb2-af73-d3f87231139f None admin] RBAC: Adding query filter params (role.id=f739a9c857cd4050896a6814c8bc0120, scope.project.id=dfeb4e06d7754aa3940fc70d5d408a30, user.id=b44229d01ea54d08a04c913b5f878f19) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:17.160542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e297a2-a408-4eb2-af73-d3f87231139f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:17.162346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e297a2-a408-4eb2-af73-d3f87231139f None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:17.168837 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 97/201] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:16 2022] GET /identity/v3/role_assignments?user.id=b44229d01ea54d08a04c913b5f878f19&scope.project.id=dfeb4e06d7754aa3940fc70d5d408a30&role.id=f739a9c857cd4050896a6814c8bc0120 => generated 610 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:17.169922 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e297a2-a408-4eb2-af73-d3f87231139f None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:17.173852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:17.221483 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:17.257519 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7623cd8-009c-42ec-a46e-1fb9833c2fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:17.258145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7623cd8-009c-42ec-a46e-1fb9833c2fcc None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:17.258435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7623cd8-009c-42ec-a46e-1fb9833c2fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:17.258712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7623cd8-009c-42ec-a46e-1fb9833c2fcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:17.304993 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e297a2-a408-4eb2-af73-d3f87231139f None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:17.348861 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d2ac34-a0cf-4518-b2bc-e7ef8c88d41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:17.349545 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2ac34-a0cf-4518-b2bc-e7ef8c88d41d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:17.349830 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2ac34-a0cf-4518-b2bc-e7ef8c88d41d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:17.350107 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2ac34-a0cf-4518-b2bc-e7ef8c88d41d None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:17.353468 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d2ac34-a0cf-4518-b2bc-e7ef8c88d41d None admin] RBAC: Authorizing `identity:get_project(project_id=51e776c25b1649278e436bb36fe083f4)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:17.354426 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d2ac34-a0cf-4518-b2bc-e7ef8c88d41d None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:17.356959 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 98/202] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:17 2022] GET /identity/v3/projects/51e776c25b1649278e436bb36fe083f4 => generated 303 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:17.362077 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d2ac34-a0cf-4518-b2bc-e7ef8c88d41d None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:17.408656 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7623cd8-009c-42ec-a46e-1fb9833c2fcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:17.410975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7623cd8-009c-42ec-a46e-1fb9833c2fcc None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:17.472026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:17.514199 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d2ac34-a0cf-4518-b2bc-e7ef8c88d41d None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:17.544937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 105/203] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:17 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:17.546127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7623cd8-009c-42ec-a46e-1fb9833c2fcc None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:17.562398 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d067c-f926-4204-9ce7-632a7b3145b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:17.563072 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d067c-f926-4204-9ce7-632a7b3145b2 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:17.563346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d067c-f926-4204-9ce7-632a7b3145b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:17.563896 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d067c-f926-4204-9ce7-632a7b3145b2 None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:17.564802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02d067c-f926-4204-9ce7-632a7b3145b2 None admin] RBAC: Authorizing `identity:get_project(project_id=a78316c83e17485ebfd6d3c10ec19d8e)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:17.566300 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02d067c-f926-4204-9ce7-632a7b3145b2 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:17.568202 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 99/204] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:17 2022] GET /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e => generated 299 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:17.568752 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d067c-f926-4204-9ce7-632a7b3145b2 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:17.577388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:17.660145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7623cd8-009c-42ec-a46e-1fb9833c2fcc None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:17.698564 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94d6cb-8bc6-4cc2-b640-d539047d666c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:17.699235 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94d6cb-8bc6-4cc2-b640-d539047d666c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:17.699579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94d6cb-8bc6-4cc2-b640-d539047d666c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:17.699870 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94d6cb-8bc6-4cc2-b640-d539047d666c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:17.700936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94d6cb-8bc6-4cc2-b640-d539047d666c None admin] RBAC: Adding query filter params (role.id=7cbff51867a7475d837afd51f6c12f11, scope.project.id=51e776c25b1649278e436bb36fe083f4, user.id=d5e07b8dbf3645eba64bf87ce3b1eb4c) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:17.701284 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94d6cb-8bc6-4cc2-b640-d539047d666c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:17.703216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94d6cb-8bc6-4cc2-b640-d539047d666c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:17.705856 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d067c-f926-4204-9ce7-632a7b3145b2 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:17.710124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 106/205] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:17 2022] GET /identity/v3/role_assignments?user.id=d5e07b8dbf3645eba64bf87ce3b1eb4c&scope.project.id=51e776c25b1649278e436bb36fe083f4&role.id=7cbff51867a7475d837afd51f6c12f11 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:17.714182 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94d6cb-8bc6-4cc2-b640-d539047d666c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:17.774833 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5e83f-3df8-46b8-be6e-e6e07f2a8421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:17.776027 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5e83f-3df8-46b8-be6e-e6e07f2a8421 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:17.778191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5e83f-3df8-46b8-be6e-e6e07f2a8421 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:17.779644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5e83f-3df8-46b8-be6e-e6e07f2a8421 None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:17.782575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5e83f-3df8-46b8-be6e-e6e07f2a8421 None admin] RBAC: Authorizing `identity:get_project(project_id=51e776c25b1649278e436bb36fe083f4)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:17.784523 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5e83f-3df8-46b8-be6e-e6e07f2a8421 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:17.788899 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 100/206] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:17 2022] GET /identity/v3/projects/51e776c25b1649278e436bb36fe083f4 => generated 303 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:17.793615 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5e83f-3df8-46b8-be6e-e6e07f2a8421 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:17.814785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:17.884339 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:17.911282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94d6cb-8bc6-4cc2-b640-d539047d666c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:17.968123 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06c2d5-6fcd-4f6b-941f-e2e05e554313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:17.969137 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06c2d5-6fcd-4f6b-941f-e2e05e554313 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:17.969626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06c2d5-6fcd-4f6b-941f-e2e05e554313 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:17.970119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06c2d5-6fcd-4f6b-941f-e2e05e554313 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:17.971108 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06c2d5-6fcd-4f6b-941f-e2e05e554313 None admin] RBAC: Adding query filter params (role.id=8724fd98145549928aa27d32c24e610a, scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e, user.id=a716df57e0834669a75c53f672b102ea) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:17.971649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06c2d5-6fcd-4f6b-941f-e2e05e554313 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:17.973554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06c2d5-6fcd-4f6b-941f-e2e05e554313 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:17.991601 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 107/207] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:17 2022] GET /identity/v3/role_assignments?user.id=a716df57e0834669a75c53f672b102ea&scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e&role.id=8724fd98145549928aa27d32c24e610a => generated 610 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:17.998840 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5e83f-3df8-46b8-be6e-e6e07f2a8421 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:18.052274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d6085-6044-4f47-940d-a4bbdd67658d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:18.053288 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d6085-6044-4f47-940d-a4bbdd67658d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:18.053776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d6085-6044-4f47-940d-a4bbdd67658d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:18.054274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d6085-6044-4f47-940d-a4bbdd67658d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:18.055283 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d6085-6044-4f47-940d-a4bbdd67658d None admin] RBAC: Adding query filter params (role.id=bb986e49c38c40bab629ff32371b9358, scope.project.id=51e776c25b1649278e436bb36fe083f4, user.id=d5e07b8dbf3645eba64bf87ce3b1eb4c) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:18.055819 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d6085-6044-4f47-940d-a4bbdd67658d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:18.057587 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d6085-6044-4f47-940d-a4bbdd67658d None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:18.067262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 101/208] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:17 2022] GET /identity/v3/role_assignments?user.id=d5e07b8dbf3645eba64bf87ce3b1eb4c&scope.project.id=51e776c25b1649278e436bb36fe083f4&role.id=bb986e49c38c40bab629ff32371b9358 => generated 260 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:18.732803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c3931-2666-4f20-9947-4bc05a1125d8 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:18.733826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c3931-2666-4f20-9947-4bc05a1125d8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:18.734340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c3931-2666-4f20-9947-4bc05a1125d8 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:18.736161 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 108/209] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:18 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:18.741851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7555640-9076-4b70-a2d7-89df2f364084 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:18.742661 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7555640-9076-4b70-a2d7-89df2f364084 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:18.743145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7555640-9076-4b70-a2d7-89df2f364084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:18.764593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-a7555640-9076-4b70-a2d7-89df2f364084 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:18.766125 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-a7555640-9076-4b70-a2d7-89df2f364084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:18.847010 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 102/210] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:18.871450 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda48d4-58f4-40c0-a7e5-eb869ebe5722 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:18.871753 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda48d4-58f4-40c0-a7e5-eb869ebe5722 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:18.872019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda48d4-58f4-40c0-a7e5-eb869ebe5722 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:18.873688 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 109/211] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:18.874387 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7555640-9076-4b70-a2d7-89df2f364084 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:18.876826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:18.881853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab79148-c740-491c-9c94-23a0e6a548c4 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:18.882144 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab79148-c740-491c-9c94-23a0e6a548c4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:18.884375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab79148-c740-491c-9c94-23a0e6a548c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:18.907839 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3ab79148-c740-491c-9c94-23a0e6a548c4 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:18.909068 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3ab79148-c740-491c-9c94-23a0e6a548c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:18.986512 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7555640-9076-4b70-a2d7-89df2f364084 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:18.988232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 110/212] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:19.012984 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab79148-c740-491c-9c94-23a0e6a548c4 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:19.030003 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49fd3d8-6234-40ba-b198-720da5058fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:19.030961 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fd3d8-6234-40ba-b198-720da5058fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:19.031400 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fd3d8-6234-40ba-b198-720da5058fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:19.032126 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fd3d8-6234-40ba-b198-720da5058fb5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:19.047070 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49fd3d8-6234-40ba-b198-720da5058fb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:19.053543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-e49fd3d8-6234-40ba-b198-720da5058fb5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:19.053543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:19.053543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:19.053543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:19.053543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:19.053543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:19.053543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:19.053543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:19.053543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:19.053543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:19.053543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:19.053543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:19.053543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:19.053543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:19.053543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:19.053543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:19.054625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:19.054625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:19.054625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:19.054625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:19.054625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:19.054625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:19.054625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:19.054625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:19.054625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:19.054625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:19.054625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:19.054625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:19.054625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:19.054625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:19.054625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:19.054625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:19.055611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:19.055611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:19.055611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:19.055611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:19.055611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:19.055611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:19.055611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:19.055611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:19.055611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:19.055611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:19.055611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:19.055611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:19.057620 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 103/213] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:19.065045 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49fd3d8-6234-40ba-b198-720da5058fb5 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:19.117680 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:19.142181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab79148-c740-491c-9c94-23a0e6a548c4 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:19.149371 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:19.182882 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e922e0b-7ec7-44c8-98e5-8867b8178976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:19.183904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e922e0b-7ec7-44c8-98e5-8867b8178976 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:19.184225 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e922e0b-7ec7-44c8-98e5-8867b8178976 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:19.184515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e922e0b-7ec7-44c8-98e5-8867b8178976 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:19.188919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e922e0b-7ec7-44c8-98e5-8867b8178976 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:19.191157 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e922e0b-7ec7-44c8-98e5-8867b8178976 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:19.194031 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 111/214] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:19 2022] GET /identity/v3/domains/default => generated 200 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:19.199851 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e922e0b-7ec7-44c8-98e5-8867b8178976 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:19.206453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49fd3d8-6234-40ba-b198-720da5058fb5 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:19.253013 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90534e2-3def-410e-83ca-045da1f84e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:19.254607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90534e2-3def-410e-83ca-045da1f84e33 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:19.254891 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90534e2-3def-410e-83ca-045da1f84e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:19.255166 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90534e2-3def-410e-83ca-045da1f84e33 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:19.255916 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90534e2-3def-410e-83ca-045da1f84e33 None admin] RBAC: Adding query filter params (name=member) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:19.256227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90534e2-3def-410e-83ca-045da1f84e33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:19.257662 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90534e2-3def-410e-83ca-045da1f84e33 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:19.271087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 104/215] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:19 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:19.271974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90534e2-3def-410e-83ca-045da1f84e33 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:19.336195 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e922e0b-7ec7-44c8-98e5-8867b8178976 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:19.385382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000df439-7092-44d8-a376-3e992d53cf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:19.385999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90534e2-3def-410e-83ca-045da1f84e33 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:19.387067 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000df439-7092-44d8-a376-3e992d53cf0d None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:19.387558 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000df439-7092-44d8-a376-3e992d53cf0d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:19.388033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000df439-7092-44d8-a376-3e992d53cf0d None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:19.390033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000df439-7092-44d8-a376-3e992d53cf0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:19.391671 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000df439-7092-44d8-a376-3e992d53cf0d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:19.415599 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:19.433152 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 112/216] 10.222.0.100 () {60 vars in 1137 bytes} [Tue Aug 2 08:57:19 2022] POST /identity/v3/users => generated 298 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 08:57:19.441382 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1833f0-18dd-4eac-b412-7e5af4daea7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:19.442158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1833f0-18dd-4eac-b412-7e5af4daea7d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:19.442449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1833f0-18dd-4eac-b412-7e5af4daea7d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:19.442726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1833f0-18dd-4eac-b412-7e5af4daea7d None admin] PATH_INFO: `/v3/users/a8ed2105c8004d199cea53a816bfd223` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:19.460324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1833f0-18dd-4eac-b412-7e5af4daea7d None admin] RBAC: Authorizing `identity:get_user(user_id=a8ed2105c8004d199cea53a816bfd223)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:19.462015 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1833f0-18dd-4eac-b412-7e5af4daea7d None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:19.464034 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 105/217] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:19 2022] GET /identity/v3/users/a8ed2105c8004d199cea53a816bfd223 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:19.470085 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000df439-7092-44d8-a376-3e992d53cf0d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:19.483437 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:19.596876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000df439-7092-44d8-a376-3e992d53cf0d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:19.638848 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92d65a-dd04-42df-8dd9-867d333f17f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:19.639492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92d65a-dd04-42df-8dd9-867d333f17f5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:19.639779 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92d65a-dd04-42df-8dd9-867d333f17f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:19.640059 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92d65a-dd04-42df-8dd9-867d333f17f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:19.714216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92d65a-dd04-42df-8dd9-867d333f17f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:19.716772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92d65a-dd04-42df-8dd9-867d333f17f5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:19.726076 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:19.826918 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 113/218] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:19 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:19.833205 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1833f0-18dd-4eac-b412-7e5af4daea7d None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:19.866497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572f00a-b6fc-4cf8-ab03-82d76e98a9ed None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:19.866791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572f00a-b6fc-4cf8-ab03-82d76e98a9ed None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:19.866978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572f00a-b6fc-4cf8-ab03-82d76e98a9ed None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:19.868369 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 114/219] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:57:19.873815 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8b6cf-11b7-440f-b061-851354d7a1fd None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:19.874111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8b6cf-11b7-440f-b061-851354d7a1fd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:19.874391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8b6cf-11b7-440f-b061-851354d7a1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:19.897006 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c2d8b6cf-11b7-440f-b061-851354d7a1fd None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:19.898393 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c2d8b6cf-11b7-440f-b061-851354d7a1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:19.968193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1833f0-18dd-4eac-b412-7e5af4daea7d None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:19.993088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 115/220] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:19 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:57:20.020281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c41e05-fa40-4f30-98ba-484f9fcf4749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:20.020985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c41e05-fa40-4f30-98ba-484f9fcf4749 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:20.021094 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8b6cf-11b7-440f-b061-851354d7a1fd None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:20.021313 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c41e05-fa40-4f30-98ba-484f9fcf4749 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:20.021591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c41e05-fa40-4f30-98ba-484f9fcf4749 None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:20.022771 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c41e05-fa40-4f30-98ba-484f9fcf4749 None admin] RBAC: Authorizing `identity:get_project(project_id=51e776c25b1649278e436bb36fe083f4)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:20.024236 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c41e05-fa40-4f30-98ba-484f9fcf4749 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:20.025933 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 106/221] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:19 2022] GET /identity/v3/projects/51e776c25b1649278e436bb36fe083f4 => generated 303 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:20.038208 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c41e05-fa40-4f30-98ba-484f9fcf4749 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:20.071668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:20.098370 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:20.154085 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c41e05-fa40-4f30-98ba-484f9fcf4749 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:20.183001 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8b6cf-11b7-440f-b061-851354d7a1fd None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:20.210338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd393c-3fe9-4252-8a2a-00baabd82c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:20.210966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd393c-3fe9-4252-8a2a-00baabd82c67 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:20.211256 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd393c-3fe9-4252-8a2a-00baabd82c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:20.211536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd393c-3fe9-4252-8a2a-00baabd82c67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:20.212456 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd393c-3fe9-4252-8a2a-00baabd82c67 None admin] RBAC: Adding query filter params (role.id=f739a9c857cd4050896a6814c8bc0120, scope.project.id=51e776c25b1649278e436bb36fe083f4, user.id=a8ed2105c8004d199cea53a816bfd223) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:20.212776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd393c-3fe9-4252-8a2a-00baabd82c67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:20.214459 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd393c-3fe9-4252-8a2a-00baabd82c67 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:20.220964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 107/222] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:20 2022] GET /identity/v3/role_assignments?user.id=a8ed2105c8004d199cea53a816bfd223&scope.project.id=51e776c25b1649278e436bb36fe083f4&role.id=f739a9c857cd4050896a6814c8bc0120 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:20.234059 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4d523-79b8-407c-b660-d54189291755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:20.235013 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4d523-79b8-407c-b660-d54189291755 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:20.235447 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4d523-79b8-407c-b660-d54189291755 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:20.235870 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4d523-79b8-407c-b660-d54189291755 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:20.260035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4d523-79b8-407c-b660-d54189291755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:20.266546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-c8b4d523-79b8-407c-b660-d54189291755 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:57:20.266546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:20.266546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:20.266546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:20.266546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:20.266546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:20.266546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:20.266546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:20.266546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:20.266546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:20.266546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:20.266546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:20.266546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:20.266546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:20.266546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:20.266546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:20.267649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:20.267649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:20.267649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:20.267649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:20.267649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:20.267649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:20.267649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:20.267649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:20.267649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:20.267649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:20.267649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:20.267649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:20.267649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:20.267649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:20.267649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:20.267649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:20.268626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:20.268626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:20.268626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:20.268626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:20.268626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:20.268626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:20.268626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:20.268626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:20.268626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:20.268626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:20.268626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:57:20.268626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:20.271170 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 116/223] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:57:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:57:20.276800 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd393c-3fe9-4252-8a2a-00baabd82c67 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:20.310156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa807c-9fdd-4535-9d20-545e23cc218e None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:20.310467 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa807c-9fdd-4535-9d20-545e23cc218e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:20.311299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa807c-9fdd-4535-9d20-545e23cc218e None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:20.312905 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 117/224] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:20 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:20.334862 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efa976-8354-41b8-8519-96f2499dbc43 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:20.335192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efa976-8354-41b8-8519-96f2499dbc43 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:20.335419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efa976-8354-41b8-8519-96f2499dbc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:20.368706 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-18efa976-8354-41b8-8519-96f2499dbc43 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:20.369935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-18efa976-8354-41b8-8519-96f2499dbc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:20.444293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd393c-3fe9-4252-8a2a-00baabd82c67 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:20.460942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:20.470689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 118/225] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:20 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:20.492310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e206f82-f813-4eb7-a585-ddf7dc584462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:20.492936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e206f82-f813-4eb7-a585-ddf7dc584462 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:20.493254 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e206f82-f813-4eb7-a585-ddf7dc584462 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:20.493533 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e206f82-f813-4eb7-a585-ddf7dc584462 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:20.494260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e206f82-f813-4eb7-a585-ddf7dc584462 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:20.494575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e206f82-f813-4eb7-a585-ddf7dc584462 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:20.496615 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e206f82-f813-4eb7-a585-ddf7dc584462 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:20.506193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 108/226] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:20 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:20.514256 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efa976-8354-41b8-8519-96f2499dbc43 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:20.542584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e206f82-f813-4eb7-a585-ddf7dc584462 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:20.561394 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:20.683756 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efa976-8354-41b8-8519-96f2499dbc43 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:20.699514 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e206f82-f813-4eb7-a585-ddf7dc584462 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:20.737828 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076041b-02fc-4578-b4b2-f74182e26fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:20.738497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076041b-02fc-4578-b4b2-f74182e26fdb None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:20.738783 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076041b-02fc-4578-b4b2-f74182e26fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:20.739065 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076041b-02fc-4578-b4b2-f74182e26fdb None admin] PATH_INFO: `/v3/users/d5e07b8dbf3645eba64bf87ce3b1eb4c` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:20.740303 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8868c-1e3d-40b3-af87-49b7f22efb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:20.740945 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8868c-1e3d-40b3-af87-49b7f22efb3f None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:20.741299 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8868c-1e3d-40b3-af87-49b7f22efb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:20.741574 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8868c-1e3d-40b3-af87-49b7f22efb3f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:20.742672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076041b-02fc-4578-b4b2-f74182e26fdb None admin] RBAC: Authorizing `identity:get_user(user_id=d5e07b8dbf3645eba64bf87ce3b1eb4c)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:20.744134 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076041b-02fc-4578-b4b2-f74182e26fdb None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:20.746284 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 119/227] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:20 2022] GET /identity/v3/users/d5e07b8dbf3645eba64bf87ce3b1eb4c => generated 284 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:20.757939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076041b-02fc-4578-b4b2-f74182e26fdb None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:20.765275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8868c-1e3d-40b3-af87-49b7f22efb3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:20.777853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-e2a8868c-1e3d-40b3-af87-49b7f22efb3f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:20.777853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:20.777853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:20.777853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:20.777853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:20.777853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:20.777853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:20.777853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:20.777853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:20.777853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:20.777853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:20.777853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:20.777853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:20.777853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:20.777853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:20.777853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:20.780182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:20.780182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:20.780182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:20.780182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:20.780182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:20.780182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:20.780182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:20.780182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:20.780182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:20.780182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:20.780182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:20.780182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:20.780182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:20.780182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:20.780182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:20.780182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:20.782132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:20.782132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:20.782132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:20.782132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:20.782132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:20.782132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:20.782132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:20.782132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:20.782132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:20.782132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:20.782132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:20.782132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:20.782132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 109/228] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:57:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:20.787158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8868c-1e3d-40b3-af87-49b7f22efb3f None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:20.859488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:20.961069 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:20.976209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076041b-02fc-4578-b4b2-f74182e26fdb None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:21.002633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8868c-1e3d-40b3-af87-49b7f22efb3f None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:21.042105 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3af9c5-cbfa-4fab-abad-f7ede9429090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:21.043585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3af9c5-cbfa-4fab-abad-f7ede9429090 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:21.044041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3af9c5-cbfa-4fab-abad-f7ede9429090 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:21.044454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3af9c5-cbfa-4fab-abad-f7ede9429090 None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:21.045809 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3af9c5-cbfa-4fab-abad-f7ede9429090 None admin] RBAC: Authorizing `identity:get_project(project_id=51e776c25b1649278e436bb36fe083f4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:21.047396 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3af9c5-cbfa-4fab-abad-f7ede9429090 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:21.049247 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7465f7-e414-439a-a5c1-3045dc3eb115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:21.049515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 120/229] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:20 2022] GET /identity/v3/projects/51e776c25b1649278e436bb36fe083f4 => generated 303 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:21.050184 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7465f7-e414-439a-a5c1-3045dc3eb115 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:21.050482 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7465f7-e414-439a-a5c1-3045dc3eb115 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:21.050765 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7465f7-e414-439a-a5c1-3045dc3eb115 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:21.051500 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7465f7-e414-439a-a5c1-3045dc3eb115 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:21.051813 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7465f7-e414-439a-a5c1-3045dc3eb115 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:21.053212 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7465f7-e414-439a-a5c1-3045dc3eb115 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:21.067103 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 110/230] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:57:20 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:21.072958 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3af9c5-cbfa-4fab-abad-f7ede9429090 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:21.073088 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7465f7-e414-439a-a5c1-3045dc3eb115 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:21.192135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7465f7-e414-439a-a5c1-3045dc3eb115 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:21.194530 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:21.230990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cb367-5e1b-4091-b1f4-380af22ff85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:21.231617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cb367-5e1b-4091-b1f4-380af22ff85f None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:21.231907 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cb367-5e1b-4091-b1f4-380af22ff85f None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:21.232423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cb367-5e1b-4091-b1f4-380af22ff85f None admin] PATH_INFO: `/v3/users/d5e07b8dbf3645eba64bf87ce3b1eb4c` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:21.233752 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597cb367-5e1b-4091-b1f4-380af22ff85f None admin] RBAC: Authorizing `identity:get_user(user_id=d5e07b8dbf3645eba64bf87ce3b1eb4c)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:21.235322 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597cb367-5e1b-4091-b1f4-380af22ff85f None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:21.241244 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 111/231] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:21 2022] GET /identity/v3/users/d5e07b8dbf3645eba64bf87ce3b1eb4c => generated 284 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:21.243008 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cb367-5e1b-4091-b1f4-380af22ff85f None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:21.299074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3af9c5-cbfa-4fab-abad-f7ede9429090 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:21.349509 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cb367-5e1b-4091-b1f4-380af22ff85f None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:21.378469 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506b539-d380-4056-a3ca-899e5cb3cf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:21.379174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506b539-d380-4056-a3ca-899e5cb3cf21 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:21.379461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506b539-d380-4056-a3ca-899e5cb3cf21 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:21.379742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506b539-d380-4056-a3ca-899e5cb3cf21 None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4/users/d5e07b8dbf3645eba64bf87ce3b1eb4c/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:21.381747 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506b539-d380-4056-a3ca-899e5cb3cf21 None admin] RBAC: Authorizing `identity:create_grant(project_id=51e776c25b1649278e436bb36fe083f4, user_id=d5e07b8dbf3645eba64bf87ce3b1eb4c, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:21.383182 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506b539-d380-4056-a3ca-899e5cb3cf21 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:21.387466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f383fe9-629c-46a7-be66-bf7bfa726df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:21.388242 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f383fe9-629c-46a7-be66-bf7bfa726df2 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:21.388588 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f383fe9-629c-46a7-be66-bf7bfa726df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:21.389002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f383fe9-629c-46a7-be66-bf7bfa726df2 None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:21.390281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f383fe9-629c-46a7-be66-bf7bfa726df2 None admin] RBAC: Authorizing `identity:get_project(project_id=51e776c25b1649278e436bb36fe083f4)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:21.391761 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f383fe9-629c-46a7-be66-bf7bfa726df2 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:21.398333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 112/232] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:21 2022] GET /identity/v3/projects/51e776c25b1649278e436bb36fe083f4 => generated 303 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:21.399466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f383fe9-629c-46a7-be66-bf7bfa726df2 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:21.421137 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 121/233] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:57:21 2022] PUT /identity/v3/projects/51e776c25b1649278e436bb36fe083f4/users/d5e07b8dbf3645eba64bf87ce3b1eb4c/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:57:21.452547 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:21.568387 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f383fe9-629c-46a7-be66-bf7bfa726df2 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:21.617922 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad016f99-14c7-4d34-af00-75848dd34a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:21.618952 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad016f99-14c7-4d34-af00-75848dd34a8c None admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:21.619233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad016f99-14c7-4d34-af00-75848dd34a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:21.619508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad016f99-14c7-4d34-af00-75848dd34a8c None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4/users/d5e07b8dbf3645eba64bf87ce3b1eb4c/roles/bb986e49c38c40bab629ff32371b9358` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:21.625992 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad016f99-14c7-4d34-af00-75848dd34a8c None admin] RBAC: Authorizing `identity:create_grant(project_id=51e776c25b1649278e436bb36fe083f4, user_id=d5e07b8dbf3645eba64bf87ce3b1eb4c, role_id=bb986e49c38c40bab629ff32371b9358)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:21.625992 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad016f99-14c7-4d34-af00-75848dd34a8c None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:21.644250 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 113/234] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:57:21 2022] PUT /identity/v3/projects/51e776c25b1649278e436bb36fe083f4/users/d5e07b8dbf3645eba64bf87ce3b1eb4c/roles/bb986e49c38c40bab629ff32371b9358 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 08:57:22.084194 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d0ed3-0d02-49c2-a512-60045b8a7210 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:22.085225 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d0ed3-0d02-49c2-a512-60045b8a7210 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:22.085724 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d0ed3-0d02-49c2-a512-60045b8a7210 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:22.087570 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 122/235] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:22 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:22.092879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac438a-aa93-465f-b62c-40ee54bfa9af None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:22.093333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac438a-aa93-465f-b62c-40ee54bfa9af None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:22.093522 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac438a-aa93-465f-b62c-40ee54bfa9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:22.112609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-d9ac438a-aa93-465f-b62c-40ee54bfa9af None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:22.114188 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d9ac438a-aa93-465f-b62c-40ee54bfa9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:22.179512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f021c2-924a-44d5-86e6-e8fee2efed33 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:22.180102 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f021c2-924a-44d5-86e6-e8fee2efed33 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:22.180611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f021c2-924a-44d5-86e6-e8fee2efed33 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:22.182314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 123/236] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:22 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:57:22.193224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 114/237] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:22 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:57:22.196056 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc91f33-36cf-41bb-923e-cdfad5fbc658 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:22.196431 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc91f33-36cf-41bb-923e-cdfad5fbc658 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:22.197438 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc91f33-36cf-41bb-923e-cdfad5fbc658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:22.214208 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac438a-aa93-465f-b62c-40ee54bfa9af None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:22.242062 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:22.242486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-fdc91f33-36cf-41bb-923e-cdfad5fbc658 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:22.243703 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fdc91f33-36cf-41bb-923e-cdfad5fbc658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:22.283007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:22.369819 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 124/238] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:22 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:22.381166 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac438a-aa93-465f-b62c-40ee54bfa9af None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:22.418732 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc91f33-36cf-41bb-923e-cdfad5fbc658 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:22.424903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ad437-d0f0-49e1-9242-eb6735f2b8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:22.425756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ad437-d0f0-49e1-9242-eb6735f2b8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:22.426205 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ad437-d0f0-49e1-9242-eb6735f2b8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:22.426643 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ad437-d0f0-49e1-9242-eb6735f2b8e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:22.454604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ad437-d0f0-49e1-9242-eb6735f2b8e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:22.463060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-cb0ad437-d0f0-49e1-9242-eb6735f2b8e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:57:22.463060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:22.463060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:22.463060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:22.463060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:22.463060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:22.463060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:22.463060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:22.463060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:22.463060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:22.463060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:22.463060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:22.463060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:22.463060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:22.463060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:22.463060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:22.464158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:22.464158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:22.464158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:22.464158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:22.464158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:22.464158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:22.464158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:22.464158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:22.464158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:22.464158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:22.464158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:22.464158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:22.464158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:22.464158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:22.464158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:22.464158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:22.466260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:22.466260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:22.466260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:22.466260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:22.466260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:22.466260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:22.466260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:22.466260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:22.466260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:22.466260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:22.466260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:57:22.466260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:22.469233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 115/239] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:57:22.471310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ad437-d0f0-49e1-9242-eb6735f2b8e2 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:22.568338 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc91f33-36cf-41bb-923e-cdfad5fbc658 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:22.607559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e16e7-637c-45e3-8f54-a30b79deaec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:22.608326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e16e7-637c-45e3-8f54-a30b79deaec5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:22.608620 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e16e7-637c-45e3-8f54-a30b79deaec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:22.608906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e16e7-637c-45e3-8f54-a30b79deaec5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:22.610105 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e16e7-637c-45e3-8f54-a30b79deaec5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:22.611578 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e16e7-637c-45e3-8f54-a30b79deaec5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:22.613650 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 125/240] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:22 2022] GET /identity/v3/domains/default => generated 200 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:22.622707 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e16e7-637c-45e3-8f54-a30b79deaec5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:22.663167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:22.674009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:22.703095 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ad437-d0f0-49e1-9242-eb6735f2b8e2 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:22.747216 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ce09e-8c53-4f49-a983-8558f80a3b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:22.747756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ce09e-8c53-4f49-a983-8558f80a3b77 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:22.748034 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ce09e-8c53-4f49-a983-8558f80a3b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:22.748308 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ce09e-8c53-4f49-a983-8558f80a3b77 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:22.749021 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ce09e-8c53-4f49-a983-8558f80a3b77 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:22.749376 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ce09e-8c53-4f49-a983-8558f80a3b77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:22.750962 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ce09e-8c53-4f49-a983-8558f80a3b77 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:22.764972 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 116/241] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:22 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:22.766286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a48fa-e44e-4b89-8dd0-1f99897ddb9e None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:22.766580 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a48fa-e44e-4b89-8dd0-1f99897ddb9e None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:22.766856 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a48fa-e44e-4b89-8dd0-1f99897ddb9e None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:22.768240 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 117/242] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:22.773012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a48fa-e44e-4b89-8dd0-1f99897ddb9e None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:22.779166 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e16e7-637c-45e3-8f54-a30b79deaec5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:22.815412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbdade-0f77-4559-b7d1-b0588426c3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:22.816041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbdade-0f77-4559-b7d1-b0588426c3e7 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:22.816329 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbdade-0f77-4559-b7d1-b0588426c3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:22.816609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbdade-0f77-4559-b7d1-b0588426c3e7 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:22.817327 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfbdade-0f77-4559-b7d1-b0588426c3e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:22.818957 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfbdade-0f77-4559-b7d1-b0588426c3e7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:22.848883 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 126/243] 10.222.0.100 () {60 vars in 1137 bytes} [Tue Aug 2 08:57:22 2022] POST /identity/v3/users => generated 294 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 08:57:22.850895 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63745346-284e-4c3a-9132-7fd8a9819426 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:22.851201 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63745346-284e-4c3a-9132-7fd8a9819426 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:22.851486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63745346-284e-4c3a-9132-7fd8a9819426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:22.876363 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-63745346-284e-4c3a-9132-7fd8a9819426 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:22.877922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-63745346-284e-4c3a-9132-7fd8a9819426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:22.951382 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a48fa-e44e-4b89-8dd0-1f99897ddb9e None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:22.969446 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 127/244] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:22 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:22.997003 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63745346-284e-4c3a-9132-7fd8a9819426 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:23.013255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:23.023675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4d6ed-d6b8-468a-b8bd-4cfe3123f1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:23.024342 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4d6ed-d6b8-468a-b8bd-4cfe3123f1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:23.024624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4d6ed-d6b8-468a-b8bd-4cfe3123f1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:23.024903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4d6ed-d6b8-468a-b8bd-4cfe3123f1d8 None admin] PATH_INFO: `/v3/users/aff2a88e6ee2430e989fc85030d8b47c` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:23.044491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4d6ed-d6b8-468a-b8bd-4cfe3123f1d8 None admin] RBAC: Authorizing `identity:get_user(user_id=aff2a88e6ee2430e989fc85030d8b47c)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:23.046202 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4d6ed-d6b8-468a-b8bd-4cfe3123f1d8 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:23.048280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 118/245] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:22 2022] GET /identity/v3/users/aff2a88e6ee2430e989fc85030d8b47c => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:23.055248 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4d6ed-d6b8-468a-b8bd-4cfe3123f1d8 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:23.061749 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:23.134872 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63745346-284e-4c3a-9132-7fd8a9819426 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:23.180052 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237e888-4fe9-49dd-80e2-bcbb7dc249e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:23.180705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237e888-4fe9-49dd-80e2-bcbb7dc249e4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:23.180992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237e888-4fe9-49dd-80e2-bcbb7dc249e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:23.181148 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4d6ed-d6b8-468a-b8bd-4cfe3123f1d8 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:23.181549 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237e888-4fe9-49dd-80e2-bcbb7dc249e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:23.197227 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237e888-4fe9-49dd-80e2-bcbb7dc249e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:23.204701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-c237e888-4fe9-49dd-80e2-bcbb7dc249e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:23.204701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:23.204701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:23.204701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:23.204701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:23.204701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:23.204701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:23.204701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:23.204701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:23.204701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:23.204701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:23.204701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:23.204701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:23.204701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:23.204701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:23.204701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:23.205637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:23.205637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:23.205637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:23.205637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:23.205637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:23.205637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:23.205637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:23.205637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:23.205637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:23.205637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:23.205637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:23.205637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:23.205637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:23.205637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:23.205637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:23.205637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:23.207242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:23.207242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:23.207242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:23.207242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:23.207242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:23.207242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:23.207242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:23.207242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:23.207242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:23.207242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:23.207242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:23.207242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:23.207242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 128/246] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:57:23.212005 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237e888-4fe9-49dd-80e2-bcbb7dc249e4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:23.228005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641fdb7-883c-4cec-a187-789d00599d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:23.230371 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641fdb7-883c-4cec-a187-789d00599d68 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:23.231305 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641fdb7-883c-4cec-a187-789d00599d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:23.231592 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641fdb7-883c-4cec-a187-789d00599d68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:23.297627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:23.323066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641fdb7-883c-4cec-a187-789d00599d68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:23.326629 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641fdb7-883c-4cec-a187-789d00599d68 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:23.330928 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:23.355855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237e888-4fe9-49dd-80e2-bcbb7dc249e4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:23.408388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a61309-6db3-4f23-8eb8-864b292a4623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:23.409029 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a61309-6db3-4f23-8eb8-864b292a4623 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:23.409339 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a61309-6db3-4f23-8eb8-864b292a4623 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:23.409615 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a61309-6db3-4f23-8eb8-864b292a4623 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:23.410401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a61309-6db3-4f23-8eb8-864b292a4623 None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:23.410712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a61309-6db3-4f23-8eb8-864b292a4623 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:23.412149 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a61309-6db3-4f23-8eb8-864b292a4623 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:23.422892 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 129/247] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:23 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:23.430476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a61309-6db3-4f23-8eb8-864b292a4623 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:23.482724 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 119/248] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:23 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:23.488649 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641fdb7-883c-4cec-a187-789d00599d68 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:23.566980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a61309-6db3-4f23-8eb8-864b292a4623 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:23.597250 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:23.600691 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641fdb7-883c-4cec-a187-789d00599d68 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:23.603459 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:23.609402 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66733a3-7436-42c1-9ac2-239030f612bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:23.610075 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66733a3-7436-42c1-9ac2-239030f612bd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:23.610364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66733a3-7436-42c1-9ac2-239030f612bd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:23.610647 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66733a3-7436-42c1-9ac2-239030f612bd None admin] PATH_INFO: `/v3/users/a8ed2105c8004d199cea53a816bfd223` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:23.611867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66733a3-7436-42c1-9ac2-239030f612bd None admin] RBAC: Authorizing `identity:get_user(user_id=a8ed2105c8004d199cea53a816bfd223)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:23.613525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66733a3-7436-42c1-9ac2-239030f612bd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:23.615234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 130/249] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:23 2022] GET /identity/v3/users/a8ed2105c8004d199cea53a816bfd223 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:23.620705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66733a3-7436-42c1-9ac2-239030f612bd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:23.643621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe5a40-94ed-4d4e-b3ee-42404abb920e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:23.644278 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe5a40-94ed-4d4e-b3ee-42404abb920e None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:23.644563 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe5a40-94ed-4d4e-b3ee-42404abb920e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:23.644843 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe5a40-94ed-4d4e-b3ee-42404abb920e None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:23.646055 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe5a40-94ed-4d4e-b3ee-42404abb920e None admin] RBAC: Authorizing `identity:get_project(project_id=51e776c25b1649278e436bb36fe083f4)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:23.647639 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe5a40-94ed-4d4e-b3ee-42404abb920e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:23.649542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 120/250] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:23 2022] GET /identity/v3/projects/51e776c25b1649278e436bb36fe083f4 => generated 303 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:23.656475 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe5a40-94ed-4d4e-b3ee-42404abb920e None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:23.734071 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66733a3-7436-42c1-9ac2-239030f612bd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:23.768923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe5a40-94ed-4d4e-b3ee-42404abb920e None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:23.784464 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdffa5-3f5d-4d52-8cbf-86aa5f0979b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:23.785259 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdffa5-3f5d-4d52-8cbf-86aa5f0979b7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:23.785548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdffa5-3f5d-4d52-8cbf-86aa5f0979b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:23.785967 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdffa5-3f5d-4d52-8cbf-86aa5f0979b7 None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:23.788916 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdffa5-3f5d-4d52-8cbf-86aa5f0979b7 None admin] RBAC: Authorizing `identity:get_project(project_id=51e776c25b1649278e436bb36fe083f4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:23.790519 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdffa5-3f5d-4d52-8cbf-86aa5f0979b7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:23.792625 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 131/251] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:23 2022] GET /identity/v3/projects/51e776c25b1649278e436bb36fe083f4 => generated 303 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:23.798382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdffa5-3f5d-4d52-8cbf-86aa5f0979b7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:23.808442 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d5f2c2-519d-4a2c-a1e1-5cfb70221eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:23.809322 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5f2c2-519d-4a2c-a1e1-5cfb70221eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:23.809605 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5f2c2-519d-4a2c-a1e1-5cfb70221eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:23.809945 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5f2c2-519d-4a2c-a1e1-5cfb70221eb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:23.810752 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d5f2c2-519d-4a2c-a1e1-5cfb70221eb1 None admin] RBAC: Adding query filter params (role.id=8724fd98145549928aa27d32c24e610a, scope.project.id=51e776c25b1649278e436bb36fe083f4, user.id=aff2a88e6ee2430e989fc85030d8b47c) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:23.811064 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d5f2c2-519d-4a2c-a1e1-5cfb70221eb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:23.812541 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d5f2c2-519d-4a2c-a1e1-5cfb70221eb1 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:23.819366 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 121/252] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:23 2022] GET /identity/v3/role_assignments?user.id=aff2a88e6ee2430e989fc85030d8b47c&scope.project.id=51e776c25b1649278e436bb36fe083f4&role.id=8724fd98145549928aa27d32c24e610a => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:23.913165 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:23.959992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdffa5-3f5d-4d52-8cbf-86aa5f0979b7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:23.997227 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396ef52-c8e0-43f7-b365-10a3390f8eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:23.997925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396ef52-c8e0-43f7-b365-10a3390f8eb6 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:23.998213 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396ef52-c8e0-43f7-b365-10a3390f8eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:23.998492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396ef52-c8e0-43f7-b365-10a3390f8eb6 None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4/users/a8ed2105c8004d199cea53a816bfd223/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:24.000485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396ef52-c8e0-43f7-b365-10a3390f8eb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=51e776c25b1649278e436bb36fe083f4, user_id=a8ed2105c8004d199cea53a816bfd223, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:24.002253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396ef52-c8e0-43f7-b365-10a3390f8eb6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:24.024920 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 132/253] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:57:23 2022] PUT /identity/v3/projects/51e776c25b1649278e436bb36fe083f4/users/a8ed2105c8004d199cea53a816bfd223/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 08:57:24.082905 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59638f8f-0a93-4e49-a07c-e50c5fb25952 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:24.083530 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59638f8f-0a93-4e49-a07c-e50c5fb25952 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:24.084011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59638f8f-0a93-4e49-a07c-e50c5fb25952 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:24.086009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 122/254] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:24 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:24.092142 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95deb75d-9bcf-49c5-9e01-cdecdd926654 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:24.092441 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95deb75d-9bcf-49c5-9e01-cdecdd926654 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:24.092722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95deb75d-9bcf-49c5-9e01-cdecdd926654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:24.114411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-95deb75d-9bcf-49c5-9e01-cdecdd926654 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:24.115990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-95deb75d-9bcf-49c5-9e01-cdecdd926654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:24.208492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 133/255] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:24 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:24.234189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59638f8f-0a93-4e49-a07c-e50c5fb25952 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:24.274654 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:24.347763 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59638f8f-0a93-4e49-a07c-e50c5fb25952 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:24.384774 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea408ae7-ac07-4b60-9644-5411f8272399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:24.385467 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea408ae7-ac07-4b60-9644-5411f8272399 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:24.385752 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea408ae7-ac07-4b60-9644-5411f8272399 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:24.386085 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea408ae7-ac07-4b60-9644-5411f8272399 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:24.397890 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea408ae7-ac07-4b60-9644-5411f8272399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:24.401230 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c35de4-6e32-4f38-b4af-daca0dded39c None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:24.401840 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c35de4-6e32-4f38-b4af-daca0dded39c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:24.402345 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c35de4-6e32-4f38-b4af-daca0dded39c None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:24.403157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-ea408ae7-ac07-4b60-9644-5411f8272399 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:24.403157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:24.403157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:24.403157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:24.403157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:24.403157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:24.403157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:24.403157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:24.403157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:24.403157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:24.403157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:24.403157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:24.403157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:24.403157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:24.403157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:24.403157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:24.404043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:24.404043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:24.404043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:24.404043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:24.404043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:24.404043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:24.404043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:24.404043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:24.404043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:24.404043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:24.404043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:24.404043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:24.404043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:24.404043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:24.404043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:24.404043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:24.404846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:24.404846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:24.404846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:24.404846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:24.404846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:24.404846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:24.404846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:24.404846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:24.404846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:24.404846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:24.404846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:24.404846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:24.404846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 123/256] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:57:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:24.408888 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 134/257] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:24 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:24.410082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea408ae7-ac07-4b60-9644-5411f8272399 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:24.428690 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f116c3-b659-4575-b051-7a1a10081259 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:24.428978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f116c3-b659-4575-b051-7a1a10081259 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:24.429296 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f116c3-b659-4575-b051-7a1a10081259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:24.467862 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-28f116c3-b659-4575-b051-7a1a10081259 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:24.468810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-28f116c3-b659-4575-b051-7a1a10081259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:24.514395 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:24.518845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea408ae7-ac07-4b60-9644-5411f8272399 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:24.557153 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543be27-ca24-45d5-aa8f-9f1b0c44e1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:24.558684 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543be27-ca24-45d5-aa8f-9f1b0c44e1b9 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:24.559167 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543be27-ca24-45d5-aa8f-9f1b0c44e1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:24.559637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543be27-ca24-45d5-aa8f-9f1b0c44e1b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:24.560573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0543be27-ca24-45d5-aa8f-9f1b0c44e1b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:24.561077 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0543be27-ca24-45d5-aa8f-9f1b0c44e1b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:24.562858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0543be27-ca24-45d5-aa8f-9f1b0c44e1b9 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:24.566610 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:24.573544 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 124/258] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:57:24 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:24.580884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543be27-ca24-45d5-aa8f-9f1b0c44e1b9 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:24.599757 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 135/259] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:24 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:57:24.643696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f116c3-b659-4575-b051-7a1a10081259 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:24.691680 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543be27-ca24-45d5-aa8f-9f1b0c44e1b9 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:24.729196 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a4153-46b5-49c5-b82c-354452c49abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:24.730719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a4153-46b5-49c5-b82c-354452c49abc None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:24.731377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a4153-46b5-49c5-b82c-354452c49abc None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:24.731856 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a4153-46b5-49c5-b82c-354452c49abc None admin] PATH_INFO: `/v3/users/d5e07b8dbf3645eba64bf87ce3b1eb4c` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:24.737381 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a4153-46b5-49c5-b82c-354452c49abc None admin] RBAC: Authorizing `identity:get_user(user_id=d5e07b8dbf3645eba64bf87ce3b1eb4c)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:24.739326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a4153-46b5-49c5-b82c-354452c49abc None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:24.741252 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 125/260] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:24 2022] GET /identity/v3/users/d5e07b8dbf3645eba64bf87ce3b1eb4c => generated 284 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:24.747415 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a4153-46b5-49c5-b82c-354452c49abc None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:24.793504 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:24.823150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f116c3-b659-4575-b051-7a1a10081259 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:24.868850 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a4153-46b5-49c5-b82c-354452c49abc None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:24.888420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c496350-c300-42dd-8039-c11ffac4fea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:24.888420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c496350-c300-42dd-8039-c11ffac4fea1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:24.888712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c496350-c300-42dd-8039-c11ffac4fea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:24.888941 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c496350-c300-42dd-8039-c11ffac4fea1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:24.906823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c496350-c300-42dd-8039-c11ffac4fea1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:24.911943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-1c496350-c300-42dd-8039-c11ffac4fea1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:57:24.911943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:24.911943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:24.911943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:24.911943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:24.911943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:24.911943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:24.911943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:24.911943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:24.911943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:24.911943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:24.911943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:24.911943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:24.911943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:24.911943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:24.911943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:24.912944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:24.912944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:24.912944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:24.912944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:24.912944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:24.912944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:24.912944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:24.912944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:24.912944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:24.912944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:24.912944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:24.912944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:24.912944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:24.912944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:24.912944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:24.912944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:24.913777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:24.913777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:24.913777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:24.913777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:24.913777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:24.913777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:24.913777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:24.913777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:24.913777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:24.913777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:24.913777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:57:24.913777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:24.921552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 136/261] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:57:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:57:24.927713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c496350-c300-42dd-8039-c11ffac4fea1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:24.945725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e148d-0f86-497f-ac42-c901db164dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:24.946368 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e148d-0f86-497f-ac42-c901db164dfa None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:24.946656 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e148d-0f86-497f-ac42-c901db164dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:24.946937 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e148d-0f86-497f-ac42-c901db164dfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:25.023904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:25.090702 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e148d-0f86-497f-ac42-c901db164dfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:25.092198 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e148d-0f86-497f-ac42-c901db164dfa None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:25.141541 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:25.159353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c496350-c300-42dd-8039-c11ffac4fea1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:25.271184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364ba9f-486e-4037-a3a3-615de1302728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:25.271954 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364ba9f-486e-4037-a3a3-615de1302728 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:25.272120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364ba9f-486e-4037-a3a3-615de1302728 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:25.272375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364ba9f-486e-4037-a3a3-615de1302728 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:25.273104 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364ba9f-486e-4037-a3a3-615de1302728 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:25.273710 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364ba9f-486e-4037-a3a3-615de1302728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:25.274818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364ba9f-486e-4037-a3a3-615de1302728 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:25.281091 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 126/262] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:24 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:25.287312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e148d-0f86-497f-ac42-c901db164dfa None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:25.298224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 137/263] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:24 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:25.304273 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364ba9f-486e-4037-a3a3-615de1302728 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:25.392502 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e148d-0f86-497f-ac42-c901db164dfa None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:25.420998 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:25.432082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934968c-720f-4d4e-b3cf-7353bbb8165c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:25.433252 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934968c-720f-4d4e-b3cf-7353bbb8165c None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:25.433748 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934968c-720f-4d4e-b3cf-7353bbb8165c None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:25.434443 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934968c-720f-4d4e-b3cf-7353bbb8165c None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:25.435951 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934968c-720f-4d4e-b3cf-7353bbb8165c None admin] RBAC: Authorizing `identity:get_project(project_id=51e776c25b1649278e436bb36fe083f4)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:25.437747 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934968c-720f-4d4e-b3cf-7353bbb8165c None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:25.440128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 127/264] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:25 2022] GET /identity/v3/projects/51e776c25b1649278e436bb36fe083f4 => generated 303 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:25.446180 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934968c-720f-4d4e-b3cf-7353bbb8165c None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:25.535181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364ba9f-486e-4037-a3a3-615de1302728 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:25.543240 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:25.550841 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934968c-720f-4d4e-b3cf-7353bbb8165c None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:25.586626 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2643a-fbec-4f0c-9e12-1713ff788b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:25.587629 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2643a-fbec-4f0c-9e12-1713ff788b97 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:25.588108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2643a-fbec-4f0c-9e12-1713ff788b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:25.588576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2643a-fbec-4f0c-9e12-1713ff788b97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:25.590231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b2643a-fbec-4f0c-9e12-1713ff788b97 None admin] RBAC: Adding query filter params (role.id=bb986e49c38c40bab629ff32371b9358, scope.project.id=51e776c25b1649278e436bb36fe083f4, user.id=d5e07b8dbf3645eba64bf87ce3b1eb4c) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:25.590772 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b2643a-fbec-4f0c-9e12-1713ff788b97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:25.592586 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b2643a-fbec-4f0c-9e12-1713ff788b97 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:25.599110 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 128/265] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:25 2022] GET /identity/v3/role_assignments?user.id=d5e07b8dbf3645eba64bf87ce3b1eb4c&scope.project.id=51e776c25b1649278e436bb36fe083f4&role.id=bb986e49c38c40bab629ff32371b9358 => generated 610 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:25.607971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12458efd-9f44-4ca1-af5b-271035b63bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:25.608699 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12458efd-9f44-4ca1-af5b-271035b63bea None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:25.608989 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12458efd-9f44-4ca1-af5b-271035b63bea None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:25.609295 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12458efd-9f44-4ca1-af5b-271035b63bea None admin] PATH_INFO: `/v3/users/d5e07b8dbf3645eba64bf87ce3b1eb4c` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:25.610593 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12458efd-9f44-4ca1-af5b-271035b63bea None admin] RBAC: Authorizing `identity:get_user(user_id=d5e07b8dbf3645eba64bf87ce3b1eb4c)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:25.612128 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12458efd-9f44-4ca1-af5b-271035b63bea None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:25.614203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 138/266] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:25 2022] GET /identity/v3/users/d5e07b8dbf3645eba64bf87ce3b1eb4c => generated 284 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:25.619750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2643a-fbec-4f0c-9e12-1713ff788b97 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:25.697611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:25.741353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2643a-fbec-4f0c-9e12-1713ff788b97 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:25.778316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43260227-976b-41e0-bdc0-7a04e4535c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:25.779285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43260227-976b-41e0-bdc0-7a04e4535c8d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:25.779792 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43260227-976b-41e0-bdc0-7a04e4535c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:25.780265 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43260227-976b-41e0-bdc0-7a04e4535c8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:25.853936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43260227-976b-41e0-bdc0-7a04e4535c8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:25.855895 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43260227-976b-41e0-bdc0-7a04e4535c8d None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:25.921544 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:25.964757 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 129/267] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:25 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:25.974379 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12458efd-9f44-4ca1-af5b-271035b63bea None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:26.117686 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12458efd-9f44-4ca1-af5b-271035b63bea None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:26.160067 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07066528-a367-425b-aca2-7028931b10d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:26.160728 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07066528-a367-425b-aca2-7028931b10d0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:26.161013 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07066528-a367-425b-aca2-7028931b10d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:26.161321 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07066528-a367-425b-aca2-7028931b10d0 None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:26.164112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07066528-a367-425b-aca2-7028931b10d0 None admin] RBAC: Authorizing `identity:get_project(project_id=51e776c25b1649278e436bb36fe083f4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:26.165729 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07066528-a367-425b-aca2-7028931b10d0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:26.168180 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 139/268] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:25 2022] GET /identity/v3/projects/51e776c25b1649278e436bb36fe083f4 => generated 303 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:26.178253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43260227-976b-41e0-bdc0-7a04e4535c8d None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:26.301757 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43260227-976b-41e0-bdc0-7a04e4535c8d None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:26.337749 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03dc96-3624-4e2f-9d00-5c413113b0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:26.338369 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03dc96-3624-4e2f-9d00-5c413113b0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:26.338653 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03dc96-3624-4e2f-9d00-5c413113b0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:26.338932 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03dc96-3624-4e2f-9d00-5c413113b0c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:26.340450 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03dc96-3624-4e2f-9d00-5c413113b0c6 None admin] RBAC: Adding query filter params (role.id=7cbff51867a7475d837afd51f6c12f11, scope.project.id=51e776c25b1649278e436bb36fe083f4, user.id=d5e07b8dbf3645eba64bf87ce3b1eb4c) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:26.340765 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03dc96-3624-4e2f-9d00-5c413113b0c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:26.342260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03dc96-3624-4e2f-9d00-5c413113b0c6 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:26.348636 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 130/269] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:26 2022] GET /identity/v3/role_assignments?user.id=d5e07b8dbf3645eba64bf87ce3b1eb4c&scope.project.id=51e776c25b1649278e436bb36fe083f4&role.id=7cbff51867a7475d837afd51f6c12f11 => generated 610 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:26.622316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9176e-6a3c-4f82-a386-638b2f6f20eb None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:26.622972 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9176e-6a3c-4f82-a386-638b2f6f20eb None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:26.622972 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9176e-6a3c-4f82-a386-638b2f6f20eb None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:26.624159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 140/270] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:26.630074 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe78917-3bf9-4e3a-b4be-94d525b5c0ec None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:26.630361 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe78917-3bf9-4e3a-b4be-94d525b5c0ec None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:26.630637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe78917-3bf9-4e3a-b4be-94d525b5c0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:26.649298 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-6fe78917-3bf9-4e3a-b4be-94d525b5c0ec None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:26.650497 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-6fe78917-3bf9-4e3a-b4be-94d525b5c0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:26.669713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64bbad-80aa-4b81-8e87-61c86251477c None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:26.669898 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:26.670468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64bbad-80aa-4b81-8e87-61c86251477c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:26.671041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64bbad-80aa-4b81-8e87-61c86251477c None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:26.672615 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 141/271] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:26 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:26.680896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a25f2-2f86-4307-8c17-f3cdbef38b9d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:26.681518 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a25f2-2f86-4307-8c17-f3cdbef38b9d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:26.682000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a25f2-2f86-4307-8c17-f3cdbef38b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:26.706980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c16a25f2-2f86-4307-8c17-f3cdbef38b9d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:26.708405 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c16a25f2-2f86-4307-8c17-f3cdbef38b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:26.725761 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 131/272] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:57:26.746608 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5368137-84c0-4421-8e17-02a10e7017e6 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:26.746865 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5368137-84c0-4421-8e17-02a10e7017e6 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:26.747305 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5368137-84c0-4421-8e17-02a10e7017e6 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:26.748805 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 132/273] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:26 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:57:26.754489 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce52b9e-958c-4fe4-87a8-e11d9386fe6e None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:26.754949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce52b9e-958c-4fe4-87a8-e11d9386fe6e None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:26.755397 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce52b9e-958c-4fe4-87a8-e11d9386fe6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:26.756528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:26.775405 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-fce52b9e-958c-4fe4-87a8-e11d9386fe6e None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:26.775586 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-fce52b9e-958c-4fe4-87a8-e11d9386fe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:26.803797 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 142/274] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:26.804812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a25f2-2f86-4307-8c17-f3cdbef38b9d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:26.853358 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 133/275] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:26.859232 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce52b9e-958c-4fe4-87a8-e11d9386fe6e None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:26.928086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a25f2-2f86-4307-8c17-f3cdbef38b9d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:26.956052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:26.963342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc433d69-a1ce-495d-943c-ff6c55311b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:26.964096 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc433d69-a1ce-495d-943c-ff6c55311b58 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:26.964451 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc433d69-a1ce-495d-943c-ff6c55311b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:26.964731 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc433d69-a1ce-495d-943c-ff6c55311b58 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:26.975338 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc433d69-a1ce-495d-943c-ff6c55311b58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:26.977525 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce52b9e-958c-4fe4-87a8-e11d9386fe6e None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:26.980946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-bc433d69-a1ce-495d-943c-ff6c55311b58 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:26.980946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:26.980946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:26.980946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:26.980946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:26.980946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:26.980946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:26.980946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:26.980946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:26.980946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:26.980946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:26.980946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:26.980946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:26.980946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:26.980946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:26.980946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:26.981861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:26.981861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:26.981861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:26.981861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:26.981861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:26.981861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:26.981861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:26.981861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:26.981861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:26.981861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:26.981861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:26.981861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:26.981861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:26.981861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:26.981861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:26.981861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:26.982677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:26.982677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:26.982677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:26.982677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:26.982677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:26.982677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:26.982677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:26.982677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:26.982677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:26.982677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:26.982677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:26.982677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:26.982677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 143/276] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:26.983366 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc433d69-a1ce-495d-943c-ff6c55311b58 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:27.017734 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:27.019036 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7639d-c4b8-4cb3-813e-bb6652f77ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:27.019503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7639d-c4b8-4cb3-813e-bb6652f77ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:27.019747 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7639d-c4b8-4cb3-813e-bb6652f77ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:27.020036 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7639d-c4b8-4cb3-813e-bb6652f77ff2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:27.021214 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a7639d-c4b8-4cb3-813e-bb6652f77ff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:27.023042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a7639d-c4b8-4cb3-813e-bb6652f77ff2 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:27.024426 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 134/277] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:26 2022] GET /identity/v3/domains/default => generated 200 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:27.025003 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7639d-c4b8-4cb3-813e-bb6652f77ff2 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:27.099796 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc433d69-a1ce-495d-943c-ff6c55311b58 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:27.136453 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e964a-4cbf-41ce-bf47-3f7fb05c0f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:27.137200 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e964a-4cbf-41ce-bf47-3f7fb05c0f90 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:27.137487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e964a-4cbf-41ce-bf47-3f7fb05c0f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:27.137763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e964a-4cbf-41ce-bf47-3f7fb05c0f90 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:27.140015 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7639d-c4b8-4cb3-813e-bb6652f77ff2 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:27.148338 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e964a-4cbf-41ce-bf47-3f7fb05c0f90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:27.153824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-022e964a-4cbf-41ce-bf47-3f7fb05c0f90 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:57:27.153824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:27.153824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:27.153824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:27.153824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:27.153824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:27.153824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:27.153824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:27.153824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:27.153824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:27.153824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:27.153824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:27.153824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:27.153824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:27.153824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:27.153824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:27.154763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:27.154763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:27.154763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:27.154763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:27.154763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:27.154763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:27.154763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:27.154763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:27.154763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:27.154763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:27.154763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:27.154763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:27.154763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:27.154763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:27.154763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:27.154763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:27.155576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:27.155576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:27.155576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:27.155576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:27.155576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:27.155576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:27.155576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:27.155576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:27.155576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:27.155576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:27.155576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:57:27.155576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:27.155576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 144/278] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:27.156224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e964a-4cbf-41ce-bf47-3f7fb05c0f90 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:27.176708 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8405744-54c2-4007-8f9f-5397a3555363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:27.177293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8405744-54c2-4007-8f9f-5397a3555363 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:27.177576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8405744-54c2-4007-8f9f-5397a3555363 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:27.177857 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8405744-54c2-4007-8f9f-5397a3555363 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:27.178566 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8405744-54c2-4007-8f9f-5397a3555363 None admin] RBAC: Adding query filter params (name=member) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:27.178875 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8405744-54c2-4007-8f9f-5397a3555363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:27.180256 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8405744-54c2-4007-8f9f-5397a3555363 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:27.189189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 135/279] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:27 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:27.190223 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f77c9-1d43-4d71-a4dd-d971cffb7128 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:27.190510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f77c9-1d43-4d71-a4dd-d971cffb7128 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:27.190791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f77c9-1d43-4d71-a4dd-d971cffb7128 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:27.192152 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 136/280] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:27 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:27.192700 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f77c9-1d43-4d71-a4dd-d971cffb7128 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:27.207628 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:27.249221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:27.258792 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e964a-4cbf-41ce-bf47-3f7fb05c0f90 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:27.293445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54138700-fefd-4abd-9603-dff3cf52e7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:27.294072 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54138700-fefd-4abd-9603-dff3cf52e7b8 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:27.294357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54138700-fefd-4abd-9603-dff3cf52e7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:27.294633 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54138700-fefd-4abd-9603-dff3cf52e7b8 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:27.294790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f77c9-1d43-4d71-a4dd-d971cffb7128 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:27.295316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54138700-fefd-4abd-9603-dff3cf52e7b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:27.296761 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54138700-fefd-4abd-9603-dff3cf52e7b8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:27.318534 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 145/281] 10.222.0.100 () {60 vars in 1137 bytes} [Tue Aug 2 08:57:27 2022] POST /identity/v3/users => generated 294 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:57:27.319353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54138700-fefd-4abd-9603-dff3cf52e7b8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:27.332946 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ef8ea-8eaa-42a4-bf52-7597d94faae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:27.333756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ef8ea-8eaa-42a4-bf52-7597d94faae0 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:27.333922 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ef8ea-8eaa-42a4-bf52-7597d94faae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:27.334194 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ef8ea-8eaa-42a4-bf52-7597d94faae0 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:27.334925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ef8ea-8eaa-42a4-bf52-7597d94faae0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:27.335236 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ef8ea-8eaa-42a4-bf52-7597d94faae0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:27.336657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ef8ea-8eaa-42a4-bf52-7597d94faae0 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:27.346774 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 137/282] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:27 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:27.352865 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d300005-f1fb-41a5-b63f-b8721c266987 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:27.353300 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d300005-f1fb-41a5-b63f-b8721c266987 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:27.353595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d300005-f1fb-41a5-b63f-b8721c266987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:27.373734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-3d300005-f1fb-41a5-b63f-b8721c266987 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:27.374972 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3d300005-f1fb-41a5-b63f-b8721c266987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:27.445728 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54138700-fefd-4abd-9603-dff3cf52e7b8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:27.454379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 138/283] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:27 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:27.466269 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d300005-f1fb-41a5-b63f-b8721c266987 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:27.474286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:27.491219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa3f5b-e823-418a-a73a-71023d1b32cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:27.491882 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa3f5b-e823-418a-a73a-71023d1b32cb None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:27.492168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa3f5b-e823-418a-a73a-71023d1b32cb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:27.492447 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa3f5b-e823-418a-a73a-71023d1b32cb None admin] PATH_INFO: `/v3/users/8fc2ba8610d64348a77e248f257cb1ec` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:27.507502 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa3f5b-e823-418a-a73a-71023d1b32cb None admin] RBAC: Authorizing `identity:get_user(user_id=8fc2ba8610d64348a77e248f257cb1ec)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:27.508973 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa3f5b-e823-418a-a73a-71023d1b32cb None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:27.510701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 146/284] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:27 2022] GET /identity/v3/users/8fc2ba8610d64348a77e248f257cb1ec => generated 294 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:27.512056 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa3f5b-e823-418a-a73a-71023d1b32cb None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:27.521393 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:27.611228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d300005-f1fb-41a5-b63f-b8721c266987 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:27.640455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa3f5b-e823-418a-a73a-71023d1b32cb None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:27.656530 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898129f7-28e3-4c83-8650-a625d62fc3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:27.657266 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898129f7-28e3-4c83-8650-a625d62fc3a6 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:27.657542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898129f7-28e3-4c83-8650-a625d62fc3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:27.657799 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898129f7-28e3-4c83-8650-a625d62fc3a6 None admin] PATH_INFO: `/v3/users/aff2a88e6ee2430e989fc85030d8b47c` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:27.659099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898129f7-28e3-4c83-8650-a625d62fc3a6 None admin] RBAC: Authorizing `identity:get_user(user_id=aff2a88e6ee2430e989fc85030d8b47c)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:27.660696 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898129f7-28e3-4c83-8650-a625d62fc3a6 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:27.662535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 139/285] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:27 2022] GET /identity/v3/users/aff2a88e6ee2430e989fc85030d8b47c => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:27.663466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898129f7-28e3-4c83-8650-a625d62fc3a6 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:27.678384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c1b33-0342-4c9c-9d70-c387d314c44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:27.679039 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c1b33-0342-4c9c-9d70-c387d314c44f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:27.679317 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c1b33-0342-4c9c-9d70-c387d314c44f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:27.679594 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c1b33-0342-4c9c-9d70-c387d314c44f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:27.690816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c1b33-0342-4c9c-9d70-c387d314c44f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:27.695907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-359c1b33-0342-4c9c-9d70-c387d314c44f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:27.695907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:27.695907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:27.695907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:27.695907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:27.695907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:27.695907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:27.695907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:27.695907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:27.695907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:27.695907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:27.695907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:27.695907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:27.695907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:27.695907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:27.695907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:27.696798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:27.696798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:27.696798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:27.696798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:27.696798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:27.696798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:27.696798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:27.696798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:27.696798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:27.696798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:27.696798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:27.696798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:27.696798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:27.696798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:27.696798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:27.696798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:27.697661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:27.697661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:27.697661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:27.697661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:27.697661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:27.697661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:27.697661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:27.697661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:27.697661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:27.697661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:27.697661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:27.697661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:27.697661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 147/286] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:27.698867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c1b33-0342-4c9c-9d70-c387d314c44f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:27.766565 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:27.787343 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:27.805793 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898129f7-28e3-4c83-8650-a625d62fc3a6 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:27.830857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c1b33-0342-4c9c-9d70-c387d314c44f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:27.845125 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4854f47-19c2-4502-aba4-4884207c93f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:27.845854 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4854f47-19c2-4502-aba4-4884207c93f2 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:27.846142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4854f47-19c2-4502-aba4-4884207c93f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:27.846418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4854f47-19c2-4502-aba4-4884207c93f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:27.847208 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4854f47-19c2-4502-aba4-4884207c93f2 None admin] RBAC: Adding query filter params (role.id=f739a9c857cd4050896a6814c8bc0120, user.id=8fc2ba8610d64348a77e248f257cb1ec, scope.system=all) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:27.847517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4854f47-19c2-4502-aba4-4884207c93f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:27.849058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4854f47-19c2-4502-aba4-4884207c93f2 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:27.854890 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 140/287] 10.222.0.100 () {56 vars in 1313 bytes} [Tue Aug 2 08:57:27 2022] GET /identity/v3/role_assignments?user.id=8fc2ba8610d64348a77e248f257cb1ec&scope.system=all&role.id=f739a9c857cd4050896a6814c8bc0120 => generated 227 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:27.855611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4854f47-19c2-4502-aba4-4884207c93f2 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:27.872804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f1cb3-ac6e-424a-93d3-079c6d1a3348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:27.873709 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f1cb3-ac6e-424a-93d3-079c6d1a3348 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:27.874089 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f1cb3-ac6e-424a-93d3-079c6d1a3348 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:27.874517 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f1cb3-ac6e-424a-93d3-079c6d1a3348 None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:27.876023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f1cb3-ac6e-424a-93d3-079c6d1a3348 None admin] RBAC: Authorizing `identity:get_project(project_id=51e776c25b1649278e436bb36fe083f4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:27.878035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f1cb3-ac6e-424a-93d3-079c6d1a3348 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:27.880201 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 148/288] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:27 2022] GET /identity/v3/projects/51e776c25b1649278e436bb36fe083f4 => generated 303 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:27.886916 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f1cb3-ac6e-424a-93d3-079c6d1a3348 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:27.972617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4854f47-19c2-4502-aba4-4884207c93f2 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:28.016155 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bc584-e75f-4efb-8599-2232f7c99d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:28.016923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bc584-e75f-4efb-8599-2232f7c99d36 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:28.017244 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bc584-e75f-4efb-8599-2232f7c99d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:28.017527 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bc584-e75f-4efb-8599-2232f7c99d36 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:28.018261 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8bc584-e75f-4efb-8599-2232f7c99d36 None admin] RBAC: Adding query filter params (name=member) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:28.018573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8bc584-e75f-4efb-8599-2232f7c99d36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:28.019981 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8bc584-e75f-4efb-8599-2232f7c99d36 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:28.022934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:28.032733 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 141/289] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:27 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:28.039840 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bc584-e75f-4efb-8599-2232f7c99d36 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:28.091731 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f1cb3-ac6e-424a-93d3-079c6d1a3348 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:28.129474 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f16e3-1a4c-42dd-93f3-ff58a300e1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:28.130506 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f16e3-1a4c-42dd-93f3-ff58a300e1f8 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:28.130990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f16e3-1a4c-42dd-93f3-ff58a300e1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:28.131464 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f16e3-1a4c-42dd-93f3-ff58a300e1f8 None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4/users/aff2a88e6ee2430e989fc85030d8b47c/roles/8724fd98145549928aa27d32c24e610a` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:28.133714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f16e3-1a4c-42dd-93f3-ff58a300e1f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=51e776c25b1649278e436bb36fe083f4, user_id=aff2a88e6ee2430e989fc85030d8b47c, role_id=8724fd98145549928aa27d32c24e610a)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:28.135368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f16e3-1a4c-42dd-93f3-ff58a300e1f8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:28.138500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:28.155851 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 149/290] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:57:27 2022] PUT /identity/v3/projects/51e776c25b1649278e436bb36fe083f4/users/aff2a88e6ee2430e989fc85030d8b47c/roles/8724fd98145549928aa27d32c24e610a => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 08:57:28.169202 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bc584-e75f-4efb-8599-2232f7c99d36 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:28.211175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247403ec-6899-4321-807a-63bff916e547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:28.212142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247403ec-6899-4321-807a-63bff916e547 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:28.212637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247403ec-6899-4321-807a-63bff916e547 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:28.213061 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247403ec-6899-4321-807a-63bff916e547 None admin] PATH_INFO: `/v3/users/a8ed2105c8004d199cea53a816bfd223` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:28.215264 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247403ec-6899-4321-807a-63bff916e547 None admin] RBAC: Authorizing `identity:get_user(user_id=a8ed2105c8004d199cea53a816bfd223)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:28.216973 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247403ec-6899-4321-807a-63bff916e547 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:28.219397 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 142/291] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:28 2022] GET /identity/v3/users/a8ed2105c8004d199cea53a816bfd223 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:28.224739 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f16e3-1a4c-42dd-93f3-ff58a300e1f8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:28.382311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f16e3-1a4c-42dd-93f3-ff58a300e1f8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:28.422454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d548c-3399-44e4-83c0-2c1a56f98bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:28.423353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d548c-3399-44e4-83c0-2c1a56f98bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:28.423769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d548c-3399-44e4-83c0-2c1a56f98bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:28.424176 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d548c-3399-44e4-83c0-2c1a56f98bf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:28.493258 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:28.503822 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d548c-3399-44e4-83c0-2c1a56f98bf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:28.505343 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d548c-3399-44e4-83c0-2c1a56f98bf2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:28.630257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 150/292] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:28 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:28.636444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247403ec-6899-4321-807a-63bff916e547 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:28.723937 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:28.772082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247403ec-6899-4321-807a-63bff916e547 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:28.815138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9359156-f79d-4a49-99ba-b1732e260202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:28.815801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9359156-f79d-4a49-99ba-b1732e260202 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:28.816088 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9359156-f79d-4a49-99ba-b1732e260202 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:28.816366 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9359156-f79d-4a49-99ba-b1732e260202 None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:28.817557 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9359156-f79d-4a49-99ba-b1732e260202 None admin] RBAC: Authorizing `identity:get_project(project_id=51e776c25b1649278e436bb36fe083f4)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:28.819015 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9359156-f79d-4a49-99ba-b1732e260202 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:28.820753 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 143/293] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:28 2022] GET /identity/v3/projects/51e776c25b1649278e436bb36fe083f4 => generated 303 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:28.828052 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d548c-3399-44e4-83c0-2c1a56f98bf2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:28.970767 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:28.981074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d548c-3399-44e4-83c0-2c1a56f98bf2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:29.031368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e75fb3-39a9-4c48-b3ce-c7bef53c29b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:29.031907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e75fb3-39a9-4c48-b3ce-c7bef53c29b5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:29.032192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e75fb3-39a9-4c48-b3ce-c7bef53c29b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:29.032467 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e75fb3-39a9-4c48-b3ce-c7bef53c29b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:29.033295 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e75fb3-39a9-4c48-b3ce-c7bef53c29b5 None admin] RBAC: Adding query filter params (role.id=f739a9c857cd4050896a6814c8bc0120, scope.project.id=51e776c25b1649278e436bb36fe083f4, user.id=a8ed2105c8004d199cea53a816bfd223) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:29.033661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e75fb3-39a9-4c48-b3ce-c7bef53c29b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:29.035084 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e75fb3-39a9-4c48-b3ce-c7bef53c29b5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:29.046406 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 151/294] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:28 2022] GET /identity/v3/role_assignments?user.id=a8ed2105c8004d199cea53a816bfd223&scope.project.id=51e776c25b1649278e436bb36fe083f4&role.id=f739a9c857cd4050896a6814c8bc0120 => generated 610 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:29.771071 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684da3d-5b76-401e-b8ad-ce2701ac3f0a None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:29.771576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684da3d-5b76-401e-b8ad-ce2701ac3f0a None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:29.771684 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684da3d-5b76-401e-b8ad-ce2701ac3f0a None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:29.773277 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 144/295] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:29.779253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c740875-c6e0-4c4e-a3e9-0c0c7a9e07e0 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:29.779545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c740875-c6e0-4c4e-a3e9-0c0c7a9e07e0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:29.780090 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c740875-c6e0-4c4e-a3e9-0c0c7a9e07e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:29.810017 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8c740875-c6e0-4c4e-a3e9-0c0c7a9e07e0 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:29.811207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8c740875-c6e0-4c4e-a3e9-0c0c7a9e07e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:29.883258 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f38c26-376c-4a9b-9261-43e16638a880 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:29.883848 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f38c26-376c-4a9b-9261-43e16638a880 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:29.884331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f38c26-376c-4a9b-9261-43e16638a880 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:29.887055 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 145/296] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:29 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:29.892778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc725210-067a-481c-976c-69b836c84843 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:29.893438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc725210-067a-481c-976c-69b836c84843 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:29.893983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc725210-067a-481c-976c-69b836c84843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:29.909621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 152/297] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:29 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:29.924304 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-fc725210-067a-481c-976c-69b836c84843 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:29.925740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-fc725210-067a-481c-976c-69b836c84843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:29.942417 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c740875-c6e0-4c4e-a3e9-0c0c7a9e07e0 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:30.049488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 146/298] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:29 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:57:30.068796 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:30.082070 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc725210-067a-481c-976c-69b836c84843 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:30.088592 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c740875-c6e0-4c4e-a3e9-0c0c7a9e07e0 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:30.129000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c424b-80b6-46c8-bc59-32eb09e2ef39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:30.129772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c424b-80b6-46c8-bc59-32eb09e2ef39 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:30.129978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c424b-80b6-46c8-bc59-32eb09e2ef39 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:30.130260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c424b-80b6-46c8-bc59-32eb09e2ef39 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:30.142364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c424b-80b6-46c8-bc59-32eb09e2ef39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:30.147183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-0d8c424b-80b6-46c8-bc59-32eb09e2ef39 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:57:30.147183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:30.147183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:30.147183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:30.147183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:30.147183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:30.147183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:30.147183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:30.147183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:30.147183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:30.147183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:30.147183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:30.147183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:30.147183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:30.147183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:30.147183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:30.148086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:30.148086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:30.148086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:30.148086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:30.148086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:30.148086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:30.148086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:30.148086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:30.148086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:30.148086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:30.148086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:30.148086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:30.148086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:30.148086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:30.148086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:30.148086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:30.148898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:30.148898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:30.148898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:30.148898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:30.148898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:30.148898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:30.148898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:30.148898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:30.148898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:30.148898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:30.148898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:57:30.148898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:30.148898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 153/299] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:30.156186 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c424b-80b6-46c8-bc59-32eb09e2ef39 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:30.179715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:30.241079 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc725210-067a-481c-976c-69b836c84843 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:30.288117 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32483c05-9d7b-4fac-b885-e10eaa651e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:30.288507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32483c05-9d7b-4fac-b885-e10eaa651e8b None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:30.288743 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32483c05-9d7b-4fac-b885-e10eaa651e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:30.289022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32483c05-9d7b-4fac-b885-e10eaa651e8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:30.290271 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32483c05-9d7b-4fac-b885-e10eaa651e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:30.292471 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32483c05-9d7b-4fac-b885-e10eaa651e8b None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:30.301532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 147/300] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:30 2022] GET /identity/v3/domains/default => generated 200 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:30.303145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32483c05-9d7b-4fac-b885-e10eaa651e8b None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:30.323147 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c424b-80b6-46c8-bc59-32eb09e2ef39 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:30.367193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10b48c-1af6-4da3-853c-060de5dda5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:30.368185 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10b48c-1af6-4da3-853c-060de5dda5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:30.368472 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10b48c-1af6-4da3-853c-060de5dda5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:30.369237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10b48c-1af6-4da3-853c-060de5dda5e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:30.369974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10b48c-1af6-4da3-853c-060de5dda5e3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:30.370291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10b48c-1af6-4da3-853c-060de5dda5e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:30.372067 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10b48c-1af6-4da3-853c-060de5dda5e3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:30.382212 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 154/301] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:30 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:30.384092 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147404d5-6118-4d3c-867d-a259adacc50f None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:30.384665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147404d5-6118-4d3c-867d-a259adacc50f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:30.385430 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147404d5-6118-4d3c-867d-a259adacc50f None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:30.387188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 155/302] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:30 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:30.392784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147404d5-6118-4d3c-867d-a259adacc50f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:30.410738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:30.443537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32483c05-9d7b-4fac-b885-e10eaa651e8b None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:30.475679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:30.483519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b559310-fa26-4826-88b4-dd2c5036470a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:30.484622 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b559310-fa26-4826-88b4-dd2c5036470a None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:30.484905 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b559310-fa26-4826-88b4-dd2c5036470a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:30.485219 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b559310-fa26-4826-88b4-dd2c5036470a None admin] PATH_INFO: `/v3/groups` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:30.485923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b559310-fa26-4826-88b4-dd2c5036470a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:30.487381 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b559310-fa26-4826-88b4-dd2c5036470a None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:30.505517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 148/303] 10.222.0.100 () {60 vars in 1139 bytes} [Tue Aug 2 08:57:30 2022] POST /identity/v3/groups => generated 230 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:57:30.507671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7712749-4be0-4076-9cc1-1a69691ed1ca None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:30.508161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7712749-4be0-4076-9cc1-1a69691ed1ca None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:30.508621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7712749-4be0-4076-9cc1-1a69691ed1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:30.543527 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-b7712749-4be0-4076-9cc1-1a69691ed1ca None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:30.545407 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b7712749-4be0-4076-9cc1-1a69691ed1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:30.547363 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147404d5-6118-4d3c-867d-a259adacc50f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:30.590156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41621c15-0572-4587-bcc1-daa5a77eba2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:30.591012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41621c15-0572-4587-bcc1-daa5a77eba2f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:30.591475 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41621c15-0572-4587-bcc1-daa5a77eba2f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:30.591829 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41621c15-0572-4587-bcc1-daa5a77eba2f None admin] PATH_INFO: `/v3/users/da4f5710e1bf489283273cadb81ee8ab` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:30.609978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41621c15-0572-4587-bcc1-daa5a77eba2f None admin] RBAC: Authorizing `identity:get_user(user_id=da4f5710e1bf489283273cadb81ee8ab)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:30.611860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41621c15-0572-4587-bcc1-daa5a77eba2f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:30.618756 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 156/304] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:30 2022] GET /identity/v3/users/da4f5710e1bf489283273cadb81ee8ab => generated 294 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:30.620045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41621c15-0572-4587-bcc1-daa5a77eba2f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:30.642353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 149/305] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:30.664287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7712749-4be0-4076-9cc1-1a69691ed1ca None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:30.703503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:30.744145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41621c15-0572-4587-bcc1-daa5a77eba2f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:30.779982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7712749-4be0-4076-9cc1-1a69691ed1ca None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:30.783413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f7890e-5b34-4eb5-8b6c-e133980f1691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:30.784044 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7890e-5b34-4eb5-8b6c-e133980f1691 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:30.784329 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7890e-5b34-4eb5-8b6c-e133980f1691 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:30.784608 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7890e-5b34-4eb5-8b6c-e133980f1691 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:30.785447 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f7890e-5b34-4eb5-8b6c-e133980f1691 None admin] RBAC: Adding query filter params (role.id=8724fd98145549928aa27d32c24e610a, user.id=da4f5710e1bf489283273cadb81ee8ab, scope.system=all) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:30.785762 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f7890e-5b34-4eb5-8b6c-e133980f1691 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:30.787200 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f7890e-5b34-4eb5-8b6c-e133980f1691 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:30.794207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 157/306] 10.222.0.100 () {56 vars in 1313 bytes} [Tue Aug 2 08:57:30 2022] GET /identity/v3/role_assignments?user.id=da4f5710e1bf489283273cadb81ee8ab&scope.system=all&role.id=8724fd98145549928aa27d32c24e610a => generated 227 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:30.795551 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b37728-7c90-479b-a5ed-f0c3874a3a5c None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:30.795835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b37728-7c90-479b-a5ed-f0c3874a3a5c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:30.796109 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b37728-7c90-479b-a5ed-f0c3874a3a5c None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:30.797677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 158/307] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:30.803538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf4483-38a9-4fab-9bd5-6832e3888384 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:30.803866 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf4483-38a9-4fab-9bd5-6832e3888384 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:30.804090 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf4483-38a9-4fab-9bd5-6832e3888384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:30.821378 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:30.826264 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e7bf4483-38a9-4fab-9bd5-6832e3888384 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:30.827572 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e7bf4483-38a9-4fab-9bd5-6832e3888384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:30.841798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480cc10-a9c8-4c95-a204-bd04e9158487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:30.843099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480cc10-a9c8-4c95-a204-bd04e9158487 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:30.843607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480cc10-a9c8-4c95-a204-bd04e9158487 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:30.844341 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480cc10-a9c8-4c95-a204-bd04e9158487 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:30.864229 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480cc10-a9c8-4c95-a204-bd04e9158487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:30.869451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-2480cc10-a9c8-4c95-a204-bd04e9158487 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:30.869451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:30.869451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:30.869451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:30.869451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:30.869451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:30.869451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:30.869451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:30.869451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:30.869451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:30.869451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:30.869451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:30.869451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:30.869451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:30.869451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:30.869451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:30.870997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:30.870997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:30.870997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:30.870997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:30.870997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:30.870997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:30.870997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:30.870997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:30.870997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:30.870997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:30.870997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:30.870997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:30.870997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:30.870997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:30.870997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:30.870997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:30.871826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:30.871826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:30.871826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:30.871826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:30.871826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:30.871826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:30.871826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:30.871826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:30.871826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:30.871826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:30.871826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:30.871826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:30.871826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 150/308] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:57:30.878306 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480cc10-a9c8-4c95-a204-bd04e9158487 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:30.927315 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 159/309] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:57:30.950448 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf4483-38a9-4fab-9bd5-6832e3888384 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:31.014067 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480cc10-a9c8-4c95-a204-bd04e9158487 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:31.041133 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:31.056344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79e7eb-c425-44f1-96d5-15c78d96f10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:31.057243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79e7eb-c425-44f1-96d5-15c78d96f10c None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:31.057726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79e7eb-c425-44f1-96d5-15c78d96f10c None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:31.058139 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79e7eb-c425-44f1-96d5-15c78d96f10c None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:31.058989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c79e7eb-c425-44f1-96d5-15c78d96f10c None admin] RBAC: Adding query filter params (name=member) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:31.059535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c79e7eb-c425-44f1-96d5-15c78d96f10c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:31.060929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c79e7eb-c425-44f1-96d5-15c78d96f10c None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:31.074886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 151/310] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:30 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:31.078079 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79e7eb-c425-44f1-96d5-15c78d96f10c None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:31.096552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf4483-38a9-4fab-9bd5-6832e3888384 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:31.130997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:31.146833 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e21bf9-97db-47c9-b463-7faf6fd2147c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:31.148045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e21bf9-97db-47c9-b463-7faf6fd2147c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:31.148521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e21bf9-97db-47c9-b463-7faf6fd2147c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:31.149069 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e21bf9-97db-47c9-b463-7faf6fd2147c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:31.163928 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e21bf9-97db-47c9-b463-7faf6fd2147c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:31.171061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-32e21bf9-97db-47c9-b463-7faf6fd2147c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:57:31.171061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:31.171061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:31.171061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:31.171061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:31.171061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:31.171061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:31.171061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:31.171061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:31.171061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:31.171061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:31.171061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:31.171061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:31.171061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:31.171061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:31.171061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:31.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:31.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:31.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:31.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:31.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:31.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:31.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:31.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:31.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:31.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:31.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:31.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:31.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:31.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:31.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:31.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:31.173162 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:31.173162 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:31.173162 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:31.173162 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:31.173162 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:31.173162 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:31.173162 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:31.173162 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:31.173162 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:31.173162 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:31.173162 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:57:31.173162 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:31.176051 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 160/311] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:57:31.182540 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e21bf9-97db-47c9-b463-7faf6fd2147c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:31.203012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79e7eb-c425-44f1-96d5-15c78d96f10c None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:31.253407 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf05fa7-c0a6-4239-beea-f507a5585d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:31.254581 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf05fa7-c0a6-4239-beea-f507a5585d92 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:31.255571 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf05fa7-c0a6-4239-beea-f507a5585d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:31.256005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf05fa7-c0a6-4239-beea-f507a5585d92 None admin] PATH_INFO: `/v3/users/8fc2ba8610d64348a77e248f257cb1ec` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:31.257787 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf05fa7-c0a6-4239-beea-f507a5585d92 None admin] RBAC: Authorizing `identity:get_user(user_id=8fc2ba8610d64348a77e248f257cb1ec)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:31.260499 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf05fa7-c0a6-4239-beea-f507a5585d92 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:31.262752 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 152/312] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:31 2022] GET /identity/v3/users/8fc2ba8610d64348a77e248f257cb1ec => generated 294 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:31.268976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf05fa7-c0a6-4239-beea-f507a5585d92 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:31.296148 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e21bf9-97db-47c9-b463-7faf6fd2147c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:31.322809 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:31.335987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9046fd-1312-4bf9-a69e-d53e48e7c93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:31.336849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9046fd-1312-4bf9-a69e-d53e48e7c93f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:31.337287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9046fd-1312-4bf9-a69e-d53e48e7c93f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:31.337706 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9046fd-1312-4bf9-a69e-d53e48e7c93f None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:31.339154 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9046fd-1312-4bf9-a69e-d53e48e7c93f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:31.339650 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9046fd-1312-4bf9-a69e-d53e48e7c93f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:31.341327 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9046fd-1312-4bf9-a69e-d53e48e7c93f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:31.350823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 161/313] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:31 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:31.356498 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9046fd-1312-4bf9-a69e-d53e48e7c93f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:31.382942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf05fa7-c0a6-4239-beea-f507a5585d92 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:31.397669 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:31.421466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03476b0e-a414-4cc0-96d5-d948afbc7969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:31.422523 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03476b0e-a414-4cc0-96d5-d948afbc7969 None admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:31.423057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03476b0e-a414-4cc0-96d5-d948afbc7969 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:31.423661 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03476b0e-a414-4cc0-96d5-d948afbc7969 None admin] PATH_INFO: `/v3/system/users/8fc2ba8610d64348a77e248f257cb1ec/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:31.425370 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03476b0e-a414-4cc0-96d5-d948afbc7969 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8fc2ba8610d64348a77e248f257cb1ec, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:31.427089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03476b0e-a414-4cc0-96d5-d948afbc7969 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:31.442518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 153/314] 10.222.0.100 () {58 vars in 1339 bytes} [Tue Aug 2 08:57:31 2022] PUT /identity/v3/system/users/8fc2ba8610d64348a77e248f257cb1ec/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 08:57:31.478729 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9046fd-1312-4bf9-a69e-d53e48e7c93f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:31.521829 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda4191-7cb9-4a1b-bc15-618f4a92f743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:31.522534 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda4191-7cb9-4a1b-bc15-618f4a92f743 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:31.522861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda4191-7cb9-4a1b-bc15-618f4a92f743 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:31.523139 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda4191-7cb9-4a1b-bc15-618f4a92f743 None admin] PATH_INFO: `/v3/users/aff2a88e6ee2430e989fc85030d8b47c` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:31.524434 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda4191-7cb9-4a1b-bc15-618f4a92f743 None admin] RBAC: Authorizing `identity:get_user(user_id=aff2a88e6ee2430e989fc85030d8b47c)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:31.526072 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda4191-7cb9-4a1b-bc15-618f4a92f743 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:31.533303 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 162/315] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:31 2022] GET /identity/v3/users/aff2a88e6ee2430e989fc85030d8b47c => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:31.535163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03476b0e-a414-4cc0-96d5-d948afbc7969 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:31.650992 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03476b0e-a414-4cc0-96d5-d948afbc7969 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:31.685521 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd9267-ce4d-4384-9675-a9a5e5390126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:31.686143 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd9267-ce4d-4384-9675-a9a5e5390126 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:31.686422 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd9267-ce4d-4384-9675-a9a5e5390126 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:31.686694 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd9267-ce4d-4384-9675-a9a5e5390126 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:31.736366 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:31.763276 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd9267-ce4d-4384-9675-a9a5e5390126 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:31.764732 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd9267-ce4d-4384-9675-a9a5e5390126 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:31.872753 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 154/316] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:31 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:31.878838 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda4191-7cb9-4a1b-bc15-618f4a92f743 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:31.935488 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:31.984728 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda4191-7cb9-4a1b-bc15-618f4a92f743 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:32.022857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd0cdf-28b4-4c95-bf3f-e5bf79de6d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:32.023507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd0cdf-28b4-4c95-bf3f-e5bf79de6d10 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:32.023789 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd0cdf-28b4-4c95-bf3f-e5bf79de6d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:32.024064 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd0cdf-28b4-4c95-bf3f-e5bf79de6d10 None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:32.025253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd0cdf-28b4-4c95-bf3f-e5bf79de6d10 None admin] RBAC: Authorizing `identity:get_project(project_id=51e776c25b1649278e436bb36fe083f4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:32.026725 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd0cdf-28b4-4c95-bf3f-e5bf79de6d10 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:32.033314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 163/317] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:31 2022] GET /identity/v3/projects/51e776c25b1649278e436bb36fe083f4 => generated 303 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:32.034761 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd9267-ce4d-4384-9675-a9a5e5390126 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:32.115794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:32.140955 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd9267-ce4d-4384-9675-a9a5e5390126 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:32.175724 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fcaef-ddd0-46b8-9017-5dc45d678c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:32.176260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fcaef-ddd0-46b8-9017-5dc45d678c9c None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:32.176536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fcaef-ddd0-46b8-9017-5dc45d678c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:32.176805 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fcaef-ddd0-46b8-9017-5dc45d678c9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:32.177642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214fcaef-ddd0-46b8-9017-5dc45d678c9c None admin] RBAC: Adding query filter params (role.id=8724fd98145549928aa27d32c24e610a, scope.project.id=51e776c25b1649278e436bb36fe083f4, user.id=aff2a88e6ee2430e989fc85030d8b47c) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:32.177950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214fcaef-ddd0-46b8-9017-5dc45d678c9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:32.179383 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214fcaef-ddd0-46b8-9017-5dc45d678c9c None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:32.192459 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 155/318] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:32 2022] GET /identity/v3/role_assignments?user.id=aff2a88e6ee2430e989fc85030d8b47c&scope.project.id=51e776c25b1649278e436bb36fe083f4&role.id=8724fd98145549928aa27d32c24e610a => generated 610 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:32.631282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f98278-bd45-43dd-824b-887c25fba4b1 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:32.631671 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f98278-bd45-43dd-824b-887c25fba4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:32.631867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f98278-bd45-43dd-824b-887c25fba4b1 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:32.633458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 164/319] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:32.639354 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c2014-c83f-4e83-b429-f040eb23bcbc None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:32.639655 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c2014-c83f-4e83-b429-f040eb23bcbc None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:32.639935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c2014-c83f-4e83-b429-f040eb23bcbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:32.658260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-c28c2014-c83f-4e83-b429-f040eb23bcbc None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:32.659457 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c28c2014-c83f-4e83-b429-f040eb23bcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:32.728620 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 156/320] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:32 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:32.750935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f98278-bd45-43dd-824b-887c25fba4b1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:32.855022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f98278-bd45-43dd-824b-887c25fba4b1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:32.875312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:32.891110 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964f207-c52a-4684-b48d-fd5a36dbb05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:32.891709 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964f207-c52a-4684-b48d-fd5a36dbb05c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:32.891986 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964f207-c52a-4684-b48d-fd5a36dbb05c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:32.892263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964f207-c52a-4684-b48d-fd5a36dbb05c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:32.893435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964f207-c52a-4684-b48d-fd5a36dbb05c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:32.894885 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964f207-c52a-4684-b48d-fd5a36dbb05c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:32.896669 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 165/321] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:32 2022] GET /identity/v3/domains/default => generated 200 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:32.905145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c2014-c83f-4e83-b429-f040eb23bcbc None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:32.919724 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c894a5-688d-4652-bd9c-5a42c464c376 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:32.919724 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c894a5-688d-4652-bd9c-5a42c464c376 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:32.919724 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c894a5-688d-4652-bd9c-5a42c464c376 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:32.920227 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 166/322] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:32 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:57:32.925254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc7072-c82f-49ee-98d4-aa977c711d76 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:32.925546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc7072-c82f-49ee-98d4-aa977c711d76 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:32.925827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc7072-c82f-49ee-98d4-aa977c711d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:32.944492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d9bc7072-c82f-49ee-98d4-aa977c711d76 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:32.945709 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d9bc7072-c82f-49ee-98d4-aa977c711d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:32.982322 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:33.017926 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c2014-c83f-4e83-b429-f040eb23bcbc None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:33.023499 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 167/323] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:32 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:33.046399 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc7072-c82f-49ee-98d4-aa977c711d76 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:33.053580 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb3e67e-1372-4e3b-8e7c-60b671547243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:33.054214 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3e67e-1372-4e3b-8e7c-60b671547243 None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:33.054499 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3e67e-1372-4e3b-8e7c-60b671547243 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:33.054776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3e67e-1372-4e3b-8e7c-60b671547243 None admin] PATH_INFO: `/v3/groups` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:33.055457 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb3e67e-1372-4e3b-8e7c-60b671547243 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:33.056922 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb3e67e-1372-4e3b-8e7c-60b671547243 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:33.085140 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 157/324] 10.222.0.100 () {60 vars in 1139 bytes} [Tue Aug 2 08:57:32 2022] POST /identity/v3/groups => generated 227 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:57:33.163974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc7072-c82f-49ee-98d4-aa977c711d76 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:33.182662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:33.202362 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281ef68-8ba9-48c9-8c50-2c7642a1a3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:33.203012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281ef68-8ba9-48c9-8c50-2c7642a1a3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:33.203293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281ef68-8ba9-48c9-8c50-2c7642a1a3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:33.203566 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281ef68-8ba9-48c9-8c50-2c7642a1a3e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:33.216076 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281ef68-8ba9-48c9-8c50-2c7642a1a3e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:33.221485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-3281ef68-8ba9-48c9-8c50-2c7642a1a3e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:57:33.221485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:33.221485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:33.221485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:33.221485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:33.221485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:33.221485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:33.221485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:33.221485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:33.221485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:33.221485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:33.221485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:33.221485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:33.221485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:33.221485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:33.221485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:33.222364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:33.222364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:33.222364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:33.222364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:33.222364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:33.222364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:33.222364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:33.222364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:33.222364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:33.222364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:33.222364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:33.222364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:33.222364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:33.222364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:33.222364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:33.222364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:33.223153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:33.223153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:33.223153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:33.223153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:33.223153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:33.223153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:33.223153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:33.223153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:33.223153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:33.223153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:33.223153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:57:33.223153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:33.226018 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 168/325] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:57:33.228532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb3e67e-1372-4e3b-8e7c-60b671547243 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:33.337685 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb3e67e-1372-4e3b-8e7c-60b671547243 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:33.369287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:33.378862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b12b85-ea20-4484-8ee0-8ff4efd04914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:33.379932 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b12b85-ea20-4484-8ee0-8ff4efd04914 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:33.380574 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b12b85-ea20-4484-8ee0-8ff4efd04914 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:33.380848 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b12b85-ea20-4484-8ee0-8ff4efd04914 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:33.381608 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b12b85-ea20-4484-8ee0-8ff4efd04914 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:33.381909 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b12b85-ea20-4484-8ee0-8ff4efd04914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:33.383282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b12b85-ea20-4484-8ee0-8ff4efd04914 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:33.397191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 158/326] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:33 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:33.399207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281ef68-8ba9-48c9-8c50-2c7642a1a3e2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:33.513920 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281ef68-8ba9-48c9-8c50-2c7642a1a3e2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:33.536851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74384f4b-cc13-4967-8f40-87f6245607a6 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:33.537153 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74384f4b-cc13-4967-8f40-87f6245607a6 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:33.537439 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74384f4b-cc13-4967-8f40-87f6245607a6 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:33.540279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 159/327] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:33.545501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc44d9-7437-4925-adcd-4b0e56811a9f None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:33.545792 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc44d9-7437-4925-adcd-4b0e56811a9f None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:33.546068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc44d9-7437-4925-adcd-4b0e56811a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:33.555438 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4411bbe-f0cf-42d8-a5e5-8b0cbcbffdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:33.556313 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4411bbe-f0cf-42d8-a5e5-8b0cbcbffdf6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:33.556905 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4411bbe-f0cf-42d8-a5e5-8b0cbcbffdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:33.557210 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4411bbe-f0cf-42d8-a5e5-8b0cbcbffdf6 None admin] PATH_INFO: `/v3/users/da4f5710e1bf489283273cadb81ee8ab` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:33.559546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4411bbe-f0cf-42d8-a5e5-8b0cbcbffdf6 None admin] RBAC: Authorizing `identity:get_user(user_id=da4f5710e1bf489283273cadb81ee8ab)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:33.561038 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4411bbe-f0cf-42d8-a5e5-8b0cbcbffdf6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:33.562843 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 169/328] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:33 2022] GET /identity/v3/users/da4f5710e1bf489283273cadb81ee8ab => generated 294 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:33.568844 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4411bbe-f0cf-42d8-a5e5-8b0cbcbffdf6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:33.573767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-49cc44d9-7437-4925-adcd-4b0e56811a9f None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:33.574923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-49cc44d9-7437-4925-adcd-4b0e56811a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:33.621375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:33.649395 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 160/329] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:33 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:33.680083 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4411bbe-f0cf-42d8-a5e5-8b0cbcbffdf6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:33.682877 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc44d9-7437-4925-adcd-4b0e56811a9f None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:33.733507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e92f4e-c3d7-4dc1-b938-c7b2a661e26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:33.734250 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e92f4e-c3d7-4dc1-b938-c7b2a661e26b None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:33.734538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e92f4e-c3d7-4dc1-b938-c7b2a661e26b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:33.734816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e92f4e-c3d7-4dc1-b938-c7b2a661e26b None admin] PATH_INFO: `/v3/system/users/da4f5710e1bf489283273cadb81ee8ab/roles/8724fd98145549928aa27d32c24e610a` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:33.736301 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e92f4e-c3d7-4dc1-b938-c7b2a661e26b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=da4f5710e1bf489283273cadb81ee8ab, role_id=8724fd98145549928aa27d32c24e610a)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:33.737712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e92f4e-c3d7-4dc1-b938-c7b2a661e26b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:33.754348 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 170/330] 10.222.0.100 () {58 vars in 1339 bytes} [Tue Aug 2 08:57:33 2022] PUT /identity/v3/system/users/da4f5710e1bf489283273cadb81ee8ab/roles/8724fd98145549928aa27d32c24e610a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:57:33.820611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc44d9-7437-4925-adcd-4b0e56811a9f None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:33.852683 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:33.868792 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ee41e-1b3c-4c6f-a52e-c75894accbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:33.869488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ee41e-1b3c-4c6f-a52e-c75894accbc6 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:33.869868 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ee41e-1b3c-4c6f-a52e-c75894accbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:33.870140 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ee41e-1b3c-4c6f-a52e-c75894accbc6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:33.882963 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ee41e-1b3c-4c6f-a52e-c75894accbc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:33.888934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-237ee41e-1b3c-4c6f-a52e-c75894accbc6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:33.888934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:33.888934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:33.888934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:33.888934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:33.888934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:33.888934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:33.888934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:33.888934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:33.888934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:33.888934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:33.888934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:33.888934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:33.888934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:33.888934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:33.888934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:33.889882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:33.889882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:33.889882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:33.889882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:33.889882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:33.889882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:33.889882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:33.889882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:33.889882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:33.889882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:33.889882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:33.889882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:33.889882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:33.889882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:33.889882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:33.889882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:33.890660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:33.890660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:33.890660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:33.890660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:33.890660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:33.890660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:33.890660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:33.890660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:33.890660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:33.890660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:33.890660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:33.890660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:33.890660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 161/331] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:57:33.897495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e92f4e-c3d7-4dc1-b938-c7b2a661e26b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:34.020644 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e92f4e-c3d7-4dc1-b938-c7b2a661e26b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:34.033039 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:34.060149 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6019e-5484-4547-96e0-e1b184b8e51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:34.060833 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6019e-5484-4547-96e0-e1b184b8e51a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:34.061159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6019e-5484-4547-96e0-e1b184b8e51a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:34.061439 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6019e-5484-4547-96e0-e1b184b8e51a None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:34.062206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6019e-5484-4547-96e0-e1b184b8e51a None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:34.063116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6019e-5484-4547-96e0-e1b184b8e51a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:34.063914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6019e-5484-4547-96e0-e1b184b8e51a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:34.073699 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 171/332] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:33 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:34.079441 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ee41e-1b3c-4c6f-a52e-c75894accbc6 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:34.191556 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ee41e-1b3c-4c6f-a52e-c75894accbc6 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:34.232496 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832e261-de50-4ab9-bb5e-00a56cbb2c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:34.233176 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832e261-de50-4ab9-bb5e-00a56cbb2c97 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:34.233467 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832e261-de50-4ab9-bb5e-00a56cbb2c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:34.233746 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832e261-de50-4ab9-bb5e-00a56cbb2c97 None admin] PATH_INFO: `/v3/users/8fc2ba8610d64348a77e248f257cb1ec` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:34.234941 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832e261-de50-4ab9-bb5e-00a56cbb2c97 None admin] RBAC: Authorizing `identity:get_user(user_id=8fc2ba8610d64348a77e248f257cb1ec)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:34.236399 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832e261-de50-4ab9-bb5e-00a56cbb2c97 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:34.238373 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 162/333] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:34 2022] GET /identity/v3/users/8fc2ba8610d64348a77e248f257cb1ec => generated 294 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:34.244233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6019e-5484-4547-96e0-e1b184b8e51a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:34.356534 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6019e-5484-4547-96e0-e1b184b8e51a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:34.393648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265aae6c-5d56-4a57-a68b-092ba35418ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:34.394278 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265aae6c-5d56-4a57-a68b-092ba35418ae None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:34.394567 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265aae6c-5d56-4a57-a68b-092ba35418ae None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:34.394848 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265aae6c-5d56-4a57-a68b-092ba35418ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:34.395643 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265aae6c-5d56-4a57-a68b-092ba35418ae None admin] RBAC: Adding query filter params (role.id=f739a9c857cd4050896a6814c8bc0120, user.id=8fc2ba8610d64348a77e248f257cb1ec, scope.system=all) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:34.395957 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265aae6c-5d56-4a57-a68b-092ba35418ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:34.397425 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265aae6c-5d56-4a57-a68b-092ba35418ae None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:34.410388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 172/334] 10.222.0.100 () {56 vars in 1313 bytes} [Tue Aug 2 08:57:34 2022] GET /identity/v3/role_assignments?user.id=8fc2ba8610d64348a77e248f257cb1ec&scope.system=all&role.id=f739a9c857cd4050896a6814c8bc0120 => generated 512 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:35.803680 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bcd0a-84e0-42e8-84c3-98301631b870 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:35.803680 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bcd0a-84e0-42e8-84c3-98301631b870 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:35.803680 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bcd0a-84e0-42e8-84c3-98301631b870 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:35.803680 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 163/335] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:35.809407 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b7d065-17e5-4ece-a786-413b58f126b6 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:35.809913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b7d065-17e5-4ece-a786-413b58f126b6 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:35.809913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b7d065-17e5-4ece-a786-413b58f126b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:35.832141 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b1b7d065-17e5-4ece-a786-413b58f126b6 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:35.833365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b1b7d065-17e5-4ece-a786-413b58f126b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:35.860151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:35.935824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 173/336] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:35 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:35.982689 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678bcd0a-84e0-42e8-84c3-98301631b870 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:36.046632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:36.107414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678bcd0a-84e0-42e8-84c3-98301631b870 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:36.152461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8764349-8deb-4add-a920-67c3a075b213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:36.153135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8764349-8deb-4add-a920-67c3a075b213 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:36.153437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8764349-8deb-4add-a920-67c3a075b213 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:36.153720 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8764349-8deb-4add-a920-67c3a075b213 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:36.167018 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8764349-8deb-4add-a920-67c3a075b213 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:36.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-c8764349-8deb-4add-a920-67c3a075b213 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:36.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:36.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:36.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:36.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:36.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:36.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:36.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:36.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:36.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:36.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:36.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:36.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:36.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:36.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:36.172145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:36.173031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:36.173031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:36.173031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:36.173031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:36.173031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:36.173031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:36.173031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:36.173031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:36.173031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:36.173031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:36.173031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:36.173031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:36.173031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:36.173031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:36.173031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:36.173031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:36.174423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:36.174423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:36.174423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:36.174423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:36.174423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:36.174423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:36.174423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:36.174423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:36.174423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:36.174423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:36.174423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:36.174423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:36.174423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 164/337] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:36.179840 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b7d065-17e5-4ece-a786-413b58f126b6 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:36.315164 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487b52f-c509-409d-9eb1-c3ff277823f6 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:36.315164 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487b52f-c509-409d-9eb1-c3ff277823f6 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:36.315439 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487b52f-c509-409d-9eb1-c3ff277823f6 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:36.316914 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 165/338] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:36.323436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b7d065-17e5-4ece-a786-413b58f126b6 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:36.323585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4e426-e506-41c1-b3a2-9ac78437b3fb None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:36.323719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4e426-e506-41c1-b3a2-9ac78437b3fb None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:36.324244 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4e426-e506-41c1-b3a2-9ac78437b3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:36.362025 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-c3e4e426-e506-41c1-b3a2-9ac78437b3fb None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:36.363156 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c3e4e426-e506-41c1-b3a2-9ac78437b3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:36.391939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c718e-f6c4-4cdc-ad24-b40d7ae22dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:36.392586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c718e-f6c4-4cdc-ad24-b40d7ae22dff None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:36.392871 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c718e-f6c4-4cdc-ad24-b40d7ae22dff None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:36.393169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c718e-f6c4-4cdc-ad24-b40d7ae22dff None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:36.393918 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c718e-f6c4-4cdc-ad24-b40d7ae22dff None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:36.394233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c718e-f6c4-4cdc-ad24-b40d7ae22dff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:36.395607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c718e-f6c4-4cdc-ad24-b40d7ae22dff None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:36.404978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 174/339] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:36 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:36.406555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b15a5-06d0-4bdf-83a2-8cb6b79a85f8 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:36.407067 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b15a5-06d0-4bdf-83a2-8cb6b79a85f8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:36.407546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b15a5-06d0-4bdf-83a2-8cb6b79a85f8 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:36.408978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 175/340] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:36.414604 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b15a5-06d0-4bdf-83a2-8cb6b79a85f8 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:36.457392 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:36.513189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 166/341] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:36 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:57:36.514570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad14100-ffbb-4c55-9065-560535defdd1 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:36.514861 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad14100-ffbb-4c55-9065-560535defdd1 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:36.515138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad14100-ffbb-4c55-9065-560535defdd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:36.550805 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-1ad14100-ffbb-4c55-9065-560535defdd1 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:36.552740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1ad14100-ffbb-4c55-9065-560535defdd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:36.611251 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b15a5-06d0-4bdf-83a2-8cb6b79a85f8 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:36.636581 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:36.671305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6f28d-69c6-4888-a19c-0c9fab98a4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:36.672210 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6f28d-69c6-4888-a19c-0c9fab98a4df None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:36.672641 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6f28d-69c6-4888-a19c-0c9fab98a4df None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:36.673061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6f28d-69c6-4888-a19c-0c9fab98a4df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:36.723233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 167/342] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:36 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:36.724285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad14100-ffbb-4c55-9065-560535defdd1 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:36.762809 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6f28d-69c6-4888-a19c-0c9fab98a4df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:36.764612 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6f28d-69c6-4888-a19c-0c9fab98a4df None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:36.796019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:36.899099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 176/343] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:36 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 08:57:36.900250 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6f28d-69c6-4888-a19c-0c9fab98a4df None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:37.031924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6f28d-69c6-4888-a19c-0c9fab98a4df None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:37.045627 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad14100-ffbb-4c55-9065-560535defdd1 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:37.077467 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:37.098662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d39ffc-1008-42d0-b0c1-c7f00ef96d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:37.099317 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d39ffc-1008-42d0-b0c1-c7f00ef96d56 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:37.099600 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d39ffc-1008-42d0-b0c1-c7f00ef96d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:37.099873 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d39ffc-1008-42d0-b0c1-c7f00ef96d56 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:37.111154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992718ad-80da-437f-8628-a4813e74a25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:37.111850 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992718ad-80da-437f-8628-a4813e74a25d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:37.112143 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992718ad-80da-437f-8628-a4813e74a25d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:37.112415 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992718ad-80da-437f-8628-a4813e74a25d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:37.121482 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d39ffc-1008-42d0-b0c1-c7f00ef96d56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:37.127255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-63d39ffc-1008-42d0-b0c1-c7f00ef96d56 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:37.127255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:37.127255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:37.127255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:37.127255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:37.127255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:37.127255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:37.127255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:37.127255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:37.127255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:37.127255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:37.127255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:37.127255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:37.127255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:37.127255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:37.127255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:37.128126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:37.128126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:37.128126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:37.128126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:37.128126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:37.128126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:37.128126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:37.128126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:37.128126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:37.128126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:37.128126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:37.128126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:37.128126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:37.128126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:37.128126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:37.128126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:37.128904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:37.128904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:37.128904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:37.128904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:37.128904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:37.128904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:37.128904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:37.128904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:37.128904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:37.128904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:37.128904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:37.128904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:37.135481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992718ad-80da-437f-8628-a4813e74a25d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:37.137199 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 177/344] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:57:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:37.138071 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d39ffc-1008-42d0-b0c1-c7f00ef96d56 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:37.142224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-992718ad-80da-437f-8628-a4813e74a25d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:37.142224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:37.142224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:37.142224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:37.142224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:37.142224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:37.142224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:37.142224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:37.142224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:37.142224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:37.142224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:37.142224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:37.142224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:37.142224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:37.142224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:37.142224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:37.144285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:37.144285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:37.144285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:37.144285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:37.144285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:37.144285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:37.144285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:37.144285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:37.144285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:37.144285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:37.144285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:37.144285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:37.144285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:37.144285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:37.144285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:37.144285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:37.146351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:37.146351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:37.146351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:37.146351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:37.146351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:37.146351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:37.146351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:37.146351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:37.146351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:37.146351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:37.146351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:37.146351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:37.153269 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 168/345] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:37.154142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992718ad-80da-437f-8628-a4813e74a25d None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:37.214235 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:37.310458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d39ffc-1008-42d0-b0c1-c7f00ef96d56 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:37.353701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ae206-ed68-454b-bf9f-04b01f0675b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:37.354652 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ae206-ed68-454b-bf9f-04b01f0675b1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:37.354912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ae206-ed68-454b-bf9f-04b01f0675b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:37.355184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ae206-ed68-454b-bf9f-04b01f0675b1 None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:37.356529 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ae206-ed68-454b-bf9f-04b01f0675b1 None admin] RBAC: Authorizing `identity:get_project(project_id=a78316c83e17485ebfd6d3c10ec19d8e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:37.358284 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ae206-ed68-454b-bf9f-04b01f0675b1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:37.360326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 178/346] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:37 2022] GET /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e => generated 299 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:37.361266 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ae206-ed68-454b-bf9f-04b01f0675b1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:37.372024 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992718ad-80da-437f-8628-a4813e74a25d None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:37.435537 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:37.458776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3923136-fff7-4c85-bc91-4b3e8a0f17ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:37.459663 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3923136-fff7-4c85-bc91-4b3e8a0f17ea None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:37.459947 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3923136-fff7-4c85-bc91-4b3e8a0f17ea None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:37.460222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3923136-fff7-4c85-bc91-4b3e8a0f17ea None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:37.460950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3923136-fff7-4c85-bc91-4b3e8a0f17ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:37.461289 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3923136-fff7-4c85-bc91-4b3e8a0f17ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:37.462688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3923136-fff7-4c85-bc91-4b3e8a0f17ea None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:37.474289 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 169/347] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:57:37 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:37.476178 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3923136-fff7-4c85-bc91-4b3e8a0f17ea None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:37.509450 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ae206-ed68-454b-bf9f-04b01f0675b1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:37.567675 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30083d9-473e-42cb-ae53-96d7b9bf8752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:37.568321 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30083d9-473e-42cb-ae53-96d7b9bf8752 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:37.569000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30083d9-473e-42cb-ae53-96d7b9bf8752 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:37.569311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30083d9-473e-42cb-ae53-96d7b9bf8752 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:37.570303 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30083d9-473e-42cb-ae53-96d7b9bf8752 None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:37.570619 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30083d9-473e-42cb-ae53-96d7b9bf8752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:37.572002 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30083d9-473e-42cb-ae53-96d7b9bf8752 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:37.589210 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 179/348] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:37 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:37.592138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec647f-5609-4c58-b28a-9ffa96ad8718 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:37.593416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec647f-5609-4c58-b28a-9ffa96ad8718 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:37.593924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec647f-5609-4c58-b28a-9ffa96ad8718 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:37.595640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 180/349] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:37 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:37.599031 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec647f-5609-4c58-b28a-9ffa96ad8718 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:37.615341 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:37.627850 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3923136-fff7-4c85-bc91-4b3e8a0f17ea None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:37.672313 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611eece2-2897-4166-8c60-589a38e6dfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:37.673320 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611eece2-2897-4166-8c60-589a38e6dfe8 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:37.673902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611eece2-2897-4166-8c60-589a38e6dfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:37.674451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611eece2-2897-4166-8c60-589a38e6dfe8 None admin] PATH_INFO: `/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:37.683667 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611eece2-2897-4166-8c60-589a38e6dfe8 None admin] RBAC: Authorizing `identity:get_group(group_id=f3b15961a0154a2abd8e3ba87e5b37b2)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:37.685478 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611eece2-2897-4166-8c60-589a38e6dfe8 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:37.687990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 170/350] 10.222.0.100 () {56 vars in 1185 bytes} [Tue Aug 2 08:57:37 2022] GET /identity/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2 => generated 227 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:37.689706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611eece2-2897-4166-8c60-589a38e6dfe8 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:37.739691 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec647f-5609-4c58-b28a-9ffa96ad8718 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:37.755850 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:37.781902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ffa50-a444-4be9-b30b-cf2435b85115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:37.782791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ffa50-a444-4be9-b30b-cf2435b85115 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:37.783211 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ffa50-a444-4be9-b30b-cf2435b85115 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:37.783622 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ffa50-a444-4be9-b30b-cf2435b85115 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:37.823533 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611eece2-2897-4166-8c60-589a38e6dfe8 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:37.869184 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8b3b4-bf14-4210-b495-64bd487c3c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:37.869900 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8b3b4-bf14-4210-b495-64bd487c3c8c None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:37.870019 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8b3b4-bf14-4210-b495-64bd487c3c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:37.870254 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8b3b4-bf14-4210-b495-64bd487c3c8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:37.881841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ffa50-a444-4be9-b30b-cf2435b85115 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:37.883346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ffa50-a444-4be9-b30b-cf2435b85115 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:37.898175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:37.947644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a8b3b4-bf14-4210-b495-64bd487c3c8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:37.949400 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a8b3b4-bf14-4210-b495-64bd487c3c8c None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:38.015632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 181/351] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:37 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:38.016812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ce0eb-20b6-489b-8bd6-8bee920aec27 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:38.017380 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ce0eb-20b6-489b-8bd6-8bee920aec27 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:38.017811 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ce0eb-20b6-489b-8bd6-8bee920aec27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:38.036196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f00ce0eb-20b6-489b-8bd6-8bee920aec27 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:38.038144 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f00ce0eb-20b6-489b-8bd6-8bee920aec27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:38.046788 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:38.069301 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 171/352] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:37 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:38.070729 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5441794-c87d-4daf-8d7c-0396df2f9b81 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:38.071180 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5441794-c87d-4daf-8d7c-0396df2f9b81 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:38.071598 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5441794-c87d-4daf-8d7c-0396df2f9b81 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:38.074231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 172/353] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:38.075200 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5441794-c87d-4daf-8d7c-0396df2f9b81 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:38.124523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 182/354] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:38 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:38.126142 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d60f0d-230e-43f7-a454-749e29f91c6c None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:38.126757 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d60f0d-230e-43f7-a454-749e29f91c6c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:38.127036 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d60f0d-230e-43f7-a454-749e29f91c6c None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:38.128827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 183/355] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:38.133123 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d60f0d-230e-43f7-a454-749e29f91c6c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:38.156467 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:38.212646 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5441794-c87d-4daf-8d7c-0396df2f9b81 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:38.253651 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f4daa-3065-4629-ab9a-623ea7d2870e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:38.254527 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f4daa-3065-4629-ab9a-623ea7d2870e None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:38.254816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f4daa-3065-4629-ab9a-623ea7d2870e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:38.255419 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f4daa-3065-4629-ab9a-623ea7d2870e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:38.255919 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f4daa-3065-4629-ab9a-623ea7d2870e None admin] RBAC: Adding query filter params (group.id=f3b15961a0154a2abd8e3ba87e5b37b2, role.id=f739a9c857cd4050896a6814c8bc0120, scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:38.256193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f4daa-3065-4629-ab9a-623ea7d2870e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:38.257660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f4daa-3065-4629-ab9a-623ea7d2870e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:38.258981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d60f0d-230e-43f7-a454-749e29f91c6c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:38.265205 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 173/356] 10.222.0.100 () {56 vars in 1381 bytes} [Tue Aug 2 08:57:38 2022] GET /identity/v3/role_assignments?group.id=f3b15961a0154a2abd8e3ba87e5b37b2&scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e&role.id=f739a9c857cd4050896a6814c8bc0120 => generated 261 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:38.266243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f4daa-3065-4629-ab9a-623ea7d2870e None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:38.302906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b7242-afbe-426b-b524-cb2b3d54a405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:38.304176 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b7242-afbe-426b-b524-cb2b3d54a405 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:38.304669 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b7242-afbe-426b-b524-cb2b3d54a405 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:38.305175 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b7242-afbe-426b-b524-cb2b3d54a405 None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:38.306695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b7242-afbe-426b-b524-cb2b3d54a405 None admin] RBAC: Authorizing `identity:get_project(project_id=a78316c83e17485ebfd6d3c10ec19d8e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:38.309176 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b7242-afbe-426b-b524-cb2b3d54a405 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:38.310919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 184/357] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:38 2022] GET /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e => generated 299 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:38.312104 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44a159-a6a9-4291-bab5-741f6e983cec None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:38.312392 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44a159-a6a9-4291-bab5-741f6e983cec None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:38.312671 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44a159-a6a9-4291-bab5-741f6e983cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:38.331925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-0d44a159-a6a9-4291-bab5-741f6e983cec None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:38.333447 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0d44a159-a6a9-4291-bab5-741f6e983cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:38.337686 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:38.414803 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f4daa-3065-4629-ab9a-623ea7d2870e None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:38.427729 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 185/358] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:38 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:38.430057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef9fa3-6f53-46d5-9161-1e3a819aac4b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:38.430557 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef9fa3-6f53-46d5-9161-1e3a819aac4b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:38.431069 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef9fa3-6f53-46d5-9161-1e3a819aac4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:38.441616 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:38.456875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-16ef9fa3-6f53-46d5-9161-1e3a819aac4b None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:38.458829 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-16ef9fa3-6f53-46d5-9161-1e3a819aac4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:38.462445 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef9d900-36b0-4b01-b1eb-ff7d6db1f27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:38.463191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9d900-36b0-4b01-b1eb-ff7d6db1f27a None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:38.463503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9d900-36b0-4b01-b1eb-ff7d6db1f27a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:38.463739 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9d900-36b0-4b01-b1eb-ff7d6db1f27a None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:38.464909 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef9d900-36b0-4b01-b1eb-ff7d6db1f27a None admin] RBAC: Authorizing `identity:get_project(project_id=51e776c25b1649278e436bb36fe083f4)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:38.466372 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef9d900-36b0-4b01-b1eb-ff7d6db1f27a None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:38.468176 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 174/359] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:38 2022] GET /identity/v3/projects/51e776c25b1649278e436bb36fe083f4 => generated 303 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:38.469475 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef9d900-36b0-4b01-b1eb-ff7d6db1f27a None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:38.544622 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 186/360] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:38 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:38.546247 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef9fa3-6f53-46d5-9161-1e3a819aac4b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:38.596730 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef9d900-36b0-4b01-b1eb-ff7d6db1f27a None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:38.641683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:38.642624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48c717-4a27-41c5-b000-02e82b4d8dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:38.643494 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48c717-4a27-41c5-b000-02e82b4d8dad None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:38.643782 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48c717-4a27-41c5-b000-02e82b4d8dad None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:38.644142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48c717-4a27-41c5-b000-02e82b4d8dad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:38.659903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48c717-4a27-41c5-b000-02e82b4d8dad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:38.665121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-cd48c717-4a27-41c5-b000-02e82b4d8dad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:57:38.665121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:38.665121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:38.665121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:38.665121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:38.665121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:38.665121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:38.665121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:38.665121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:38.665121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:38.665121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:38.665121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:38.665121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:38.665121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:38.665121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:38.665121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:38.665974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:38.665974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:38.665974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:38.665974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:38.665974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:38.665974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:38.665974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:38.665974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:38.665974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:38.665974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:38.665974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:38.665974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:38.665974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:38.665974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:38.665974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:38.665974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:38.667758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:38.667758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:38.667758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:38.667758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:38.667758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:38.667758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:38.667758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:38.667758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:38.667758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:38.667758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:38.667758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:57:38.667758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:38.667758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 175/361] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:38.667758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48c717-4a27-41c5-b000-02e82b4d8dad None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:38.692187 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef9fa3-6f53-46d5-9161-1e3a819aac4b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:38.742365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:38.749964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb77f1f-321e-47ee-9954-7bf9ba0024bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:38.750718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb77f1f-321e-47ee-9954-7bf9ba0024bd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:38.751004 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb77f1f-321e-47ee-9954-7bf9ba0024bd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:38.753040 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb77f1f-321e-47ee-9954-7bf9ba0024bd None admin] PATH_INFO: `/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:38.754509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb77f1f-321e-47ee-9954-7bf9ba0024bd None admin] RBAC: Authorizing `identity:get_group(group_id=f3b15961a0154a2abd8e3ba87e5b37b2)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:38.755949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb77f1f-321e-47ee-9954-7bf9ba0024bd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:38.758140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 187/362] 10.222.0.100 () {56 vars in 1185 bytes} [Tue Aug 2 08:57:38 2022] GET /identity/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2 => generated 227 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:38.758460 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb77f1f-321e-47ee-9954-7bf9ba0024bd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:38.807721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48c717-4a27-41c5-b000-02e82b4d8dad None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:38.850824 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40ebc1-b252-4e84-8f9f-cec7900a9a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:38.851477 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40ebc1-b252-4e84-8f9f-cec7900a9a73 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:38.851765 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40ebc1-b252-4e84-8f9f-cec7900a9a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:38.852045 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40ebc1-b252-4e84-8f9f-cec7900a9a73 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:38.872463 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40ebc1-b252-4e84-8f9f-cec7900a9a73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:38.878469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-0a40ebc1-b252-4e84-8f9f-cec7900a9a73 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:38.878469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:38.878469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:38.878469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:38.878469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:38.878469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:38.878469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:38.878469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:38.878469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:38.878469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:38.878469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:38.878469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:38.878469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:38.878469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:38.878469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:38.878469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:38.879346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:38.879346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:38.879346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:38.879346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:38.879346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:38.879346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:38.879346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:38.879346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:38.879346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:38.879346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:38.879346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:38.879346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:38.879346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:38.879346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:38.879346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:38.879346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:38.880154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:38.880154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:38.880154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:38.880154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:38.880154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:38.880154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:38.880154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:38.880154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:38.880154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:38.880154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:38.880154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:38.880154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:38.880154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 176/363] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:57:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:38.880807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40ebc1-b252-4e84-8f9f-cec7900a9a73 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:38.907920 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb77f1f-321e-47ee-9954-7bf9ba0024bd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:38.970784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad909a-9285-4933-9d52-fd28f2fa88b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:38.971464 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad909a-9285-4933-9d52-fd28f2fa88b8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:38.971751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad909a-9285-4933-9d52-fd28f2fa88b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:38.972021 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad909a-9285-4933-9d52-fd28f2fa88b8 None admin] PATH_INFO: `/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:38.974192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad909a-9285-4933-9d52-fd28f2fa88b8 None admin] RBAC: Authorizing `identity:get_group(group_id=f3b15961a0154a2abd8e3ba87e5b37b2)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:38.975896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad909a-9285-4933-9d52-fd28f2fa88b8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:38.978630 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 188/364] 10.222.0.100 () {56 vars in 1185 bytes} [Tue Aug 2 08:57:38 2022] GET /identity/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2 => generated 227 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:38.980564 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad909a-9285-4933-9d52-fd28f2fa88b8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:38.990890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:39.036047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40ebc1-b252-4e84-8f9f-cec7900a9a73 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:39.047212 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:39.091545 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c9d3b-14c4-4bd7-ad1e-8368b8176959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:39.093267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c9d3b-14c4-4bd7-ad1e-8368b8176959 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:39.094222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c9d3b-14c4-4bd7-ad1e-8368b8176959 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:39.097446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c9d3b-14c4-4bd7-ad1e-8368b8176959 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:39.109740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c9d3b-14c4-4bd7-ad1e-8368b8176959 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:39.117326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-c80c9d3b-14c4-4bd7-ad1e-8368b8176959 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:57:39.117326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:39.117326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:39.117326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:39.117326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:39.117326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:39.117326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:39.117326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:39.117326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:39.117326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:39.117326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:39.117326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:39.117326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:39.117326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:39.117326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:39.117326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:39.118333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:39.118333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:39.118333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:39.118333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:39.118333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:39.118333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:39.118333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:39.118333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:39.118333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:39.118333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:39.118333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:39.118333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:39.118333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:39.118333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:39.118333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:39.118333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:39.119120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:39.119120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:39.119120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:39.119120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:39.119120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:39.119120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:39.119120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:39.119120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:39.119120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:39.119120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:39.119120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:57:39.119120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:39.119772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad909a-9285-4933-9d52-fd28f2fa88b8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:39.121401 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 177/365] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:57:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:39.122217 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c9d3b-14c4-4bd7-ad1e-8368b8176959 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:39.163872 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581223cd-fb8e-4377-b1e8-4c6647ab8ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:39.164562 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581223cd-fb8e-4377-b1e8-4c6647ab8ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:39.164851 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581223cd-fb8e-4377-b1e8-4c6647ab8ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:39.165178 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581223cd-fb8e-4377-b1e8-4c6647ab8ee8 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:39.165906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581223cd-fb8e-4377-b1e8-4c6647ab8ee8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:39.166220 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581223cd-fb8e-4377-b1e8-4c6647ab8ee8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:39.167627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581223cd-fb8e-4377-b1e8-4c6647ab8ee8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:39.181435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 189/366] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:38 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:39.181435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581223cd-fb8e-4377-b1e8-4c6647ab8ee8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:39.267666 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c9d3b-14c4-4bd7-ad1e-8368b8176959 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:39.271389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:39.319501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a36181-a928-4b09-8802-04a8c8de2f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:39.320046 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a36181-a928-4b09-8802-04a8c8de2f9e None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:39.320332 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a36181-a928-4b09-8802-04a8c8de2f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:39.320612 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a36181-a928-4b09-8802-04a8c8de2f9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:39.321534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a36181-a928-4b09-8802-04a8c8de2f9e None admin] RBAC: Adding query filter params (group.id=f3b15961a0154a2abd8e3ba87e5b37b2, role.id=bb986e49c38c40bab629ff32371b9358, scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:39.321879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a36181-a928-4b09-8802-04a8c8de2f9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:39.323403 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581223cd-fb8e-4377-b1e8-4c6647ab8ee8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:39.323525 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a36181-a928-4b09-8802-04a8c8de2f9e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:39.330398 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 178/367] 10.222.0.100 () {56 vars in 1381 bytes} [Tue Aug 2 08:57:39 2022] GET /identity/v3/role_assignments?group.id=f3b15961a0154a2abd8e3ba87e5b37b2&scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e&role.id=bb986e49c38c40bab629ff32371b9358 => generated 261 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:39.331044 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a36181-a928-4b09-8802-04a8c8de2f9e None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:39.365892 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce46809c-27ec-4945-93a8-004c7ce6df55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:39.366944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce46809c-27ec-4945-93a8-004c7ce6df55 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:39.367452 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce46809c-27ec-4945-93a8-004c7ce6df55 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:39.367923 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce46809c-27ec-4945-93a8-004c7ce6df55 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:39.368969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce46809c-27ec-4945-93a8-004c7ce6df55 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:39.369662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce46809c-27ec-4945-93a8-004c7ce6df55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:39.371242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce46809c-27ec-4945-93a8-004c7ce6df55 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:39.372885 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:39.386857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 190/368] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:57:39 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:39.387729 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce46809c-27ec-4945-93a8-004c7ce6df55 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:39.475166 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a36181-a928-4b09-8802-04a8c8de2f9e None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:39.517760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc21f0c-ef16-4dea-87da-bfa4036542fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:39.518512 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc21f0c-ef16-4dea-87da-bfa4036542fa None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:39.518801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc21f0c-ef16-4dea-87da-bfa4036542fa None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:39.519197 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc21f0c-ef16-4dea-87da-bfa4036542fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:39.519994 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc21f0c-ef16-4dea-87da-bfa4036542fa None admin] RBAC: Adding query filter params (group.id=f3b15961a0154a2abd8e3ba87e5b37b2, role.id=f739a9c857cd4050896a6814c8bc0120, scope.project.id=51e776c25b1649278e436bb36fe083f4) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:39.520321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc21f0c-ef16-4dea-87da-bfa4036542fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:39.522185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc21f0c-ef16-4dea-87da-bfa4036542fa None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:39.522708 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce46809c-27ec-4945-93a8-004c7ce6df55 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:39.534258 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 179/369] 10.222.0.100 () {56 vars in 1381 bytes} [Tue Aug 2 08:57:39 2022] GET /identity/v3/role_assignments?group.id=f3b15961a0154a2abd8e3ba87e5b37b2&scope.project.id=51e776c25b1649278e436bb36fe083f4&role.id=f739a9c857cd4050896a6814c8bc0120 => generated 261 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:39.534258 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc21f0c-ef16-4dea-87da-bfa4036542fa None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:39.586696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:39.599797 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf354b-9d47-49b9-8bcf-a641dde625af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:39.601506 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf354b-9d47-49b9-8bcf-a641dde625af None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:39.601803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf354b-9d47-49b9-8bcf-a641dde625af None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:39.602083 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf354b-9d47-49b9-8bcf-a641dde625af None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:39.602817 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf354b-9d47-49b9-8bcf-a641dde625af None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:39.603129 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf354b-9d47-49b9-8bcf-a641dde625af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:39.606012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf354b-9d47-49b9-8bcf-a641dde625af None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:39.624355 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 191/370] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:39 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:39.625053 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf354b-9d47-49b9-8bcf-a641dde625af None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:39.726156 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:39.733523 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc21f0c-ef16-4dea-87da-bfa4036542fa None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:39.771526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ef20d-7495-4eb1-8817-752c5a1e3ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:39.772529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ef20d-7495-4eb1-8817-752c5a1e3ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:39.773093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ef20d-7495-4eb1-8817-752c5a1e3ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:39.773667 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ef20d-7495-4eb1-8817-752c5a1e3ad5 None admin] PATH_INFO: `/v3/users/da4f5710e1bf489283273cadb81ee8ab` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:39.775738 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ef20d-7495-4eb1-8817-752c5a1e3ad5 None admin] RBAC: Authorizing `identity:get_user(user_id=da4f5710e1bf489283273cadb81ee8ab)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:39.778464 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ef20d-7495-4eb1-8817-752c5a1e3ad5 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:39.779827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf354b-9d47-49b9-8bcf-a641dde625af None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:39.780878 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 180/371] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:57:39 2022] GET /identity/v3/users/da4f5710e1bf489283273cadb81ee8ab => generated 294 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:39.781950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ef20d-7495-4eb1-8817-752c5a1e3ad5 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:39.863963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f4d3e-6dda-4d94-ac33-aa9b1770f240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:39.864515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f4d3e-6dda-4d94-ac33-aa9b1770f240 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:39.864796 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f4d3e-6dda-4d94-ac33-aa9b1770f240 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:39.865069 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f4d3e-6dda-4d94-ac33-aa9b1770f240 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:39.890475 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ef20d-7495-4eb1-8817-752c5a1e3ad5 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:39.929699 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c06a51-1a35-4be6-9cc5-7271a1fe30ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:39.930645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c06a51-1a35-4be6-9cc5-7271a1fe30ef None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:39.931148 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c06a51-1a35-4be6-9cc5-7271a1fe30ef None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:39.931623 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c06a51-1a35-4be6-9cc5-7271a1fe30ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:39.979573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:40.006177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c06a51-1a35-4be6-9cc5-7271a1fe30ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:40.008056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c06a51-1a35-4be6-9cc5-7271a1fe30ef None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:40.050131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:40.085983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f4d3e-6dda-4d94-ac33-aa9b1770f240 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:40.087545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f4d3e-6dda-4d94-ac33-aa9b1770f240 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:40.118350 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 181/372] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:39 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:40.125391 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c06a51-1a35-4be6-9cc5-7271a1fe30ef None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:40.234672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:40.287281 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 192/373] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:39 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:40.288225 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f4d3e-6dda-4d94-ac33-aa9b1770f240 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:40.302377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c06a51-1a35-4be6-9cc5-7271a1fe30ef None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:40.366467 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875665b1-1a8f-407f-b53f-44ae75f44cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:40.367091 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875665b1-1a8f-407f-b53f-44ae75f44cba None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:40.367373 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875665b1-1a8f-407f-b53f-44ae75f44cba None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:40.367649 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875665b1-1a8f-407f-b53f-44ae75f44cba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:40.368551 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875665b1-1a8f-407f-b53f-44ae75f44cba None admin] RBAC: Adding query filter params (role.id=8724fd98145549928aa27d32c24e610a, user.id=da4f5710e1bf489283273cadb81ee8ab, scope.system=all) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:40.368814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875665b1-1a8f-407f-b53f-44ae75f44cba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:40.371479 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875665b1-1a8f-407f-b53f-44ae75f44cba None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:40.397139 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 182/374] 10.222.0.100 () {56 vars in 1313 bytes} [Tue Aug 2 08:57:40 2022] GET /identity/v3/role_assignments?user.id=da4f5710e1bf489283273cadb81ee8ab&scope.system=all&role.id=8724fd98145549928aa27d32c24e610a => generated 512 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:40.398251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875665b1-1a8f-407f-b53f-44ae75f44cba None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:40.467391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:40.488798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f4d3e-6dda-4d94-ac33-aa9b1770f240 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:40.561386 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28810f7-1639-4522-8c76-954ee7fe4ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:40.562386 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28810f7-1639-4522-8c76-954ee7fe4ccf None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:40.562810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28810f7-1639-4522-8c76-954ee7fe4ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:40.563228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28810f7-1639-4522-8c76-954ee7fe4ccf None admin] PATH_INFO: `/v3/projects/1b02845ab43a49ada6db1e8f52cf0026` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:40.570162 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28810f7-1639-4522-8c76-954ee7fe4ccf None admin] RBAC: Authorizing `identity:get_project(project_id=1b02845ab43a49ada6db1e8f52cf0026)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:40.572143 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28810f7-1639-4522-8c76-954ee7fe4ccf None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:40.575730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 193/375] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:40 2022] GET /identity/v3/projects/1b02845ab43a49ada6db1e8f52cf0026 => generated 345 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:40.577408 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5810a0-258d-4df3-a8f8-7e9956b010c0 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:40.577904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5810a0-258d-4df3-a8f8-7e9956b010c0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:40.578379 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5810a0-258d-4df3-a8f8-7e9956b010c0 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:40.583202 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875665b1-1a8f-407f-b53f-44ae75f44cba None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:40.583517 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 194/376] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:40 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:40.584637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5810a0-258d-4df3-a8f8-7e9956b010c0 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:40.629620 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:40.644394 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484b520-16bc-4347-93d0-b3f320eabb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:40.645509 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484b520-16bc-4347-93d0-b3f320eabb07 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:40.645936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484b520-16bc-4347-93d0-b3f320eabb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:40.649988 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484b520-16bc-4347-93d0-b3f320eabb07 None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:40.652387 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484b520-16bc-4347-93d0-b3f320eabb07 None admin] RBAC: Authorizing `identity:get_project(project_id=51e776c25b1649278e436bb36fe083f4)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:40.652972 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484b520-16bc-4347-93d0-b3f320eabb07 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:40.659442 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 183/377] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:40 2022] GET /identity/v3/projects/51e776c25b1649278e436bb36fe083f4 => generated 303 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:40.660511 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f24f2-529d-4961-9455-1d57cce2f97b None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:40.660798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f24f2-529d-4961-9455-1d57cce2f97b None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:40.661172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f24f2-529d-4961-9455-1d57cce2f97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:40.684742 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-c04f24f2-529d-4961-9455-1d57cce2f97b None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:40.685993 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c04f24f2-529d-4961-9455-1d57cce2f97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:40.752948 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5810a0-258d-4df3-a8f8-7e9956b010c0 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:40.768384 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 184/378] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:40 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:40.769545 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f24f2-529d-4961-9455-1d57cce2f97b None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:40.798777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e60463-23a9-4c45-ab28-77782a49285b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:40.799435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e60463-23a9-4c45-ab28-77782a49285b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:40.799719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e60463-23a9-4c45-ab28-77782a49285b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:40.799996 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e60463-23a9-4c45-ab28-77782a49285b None admin] PATH_INFO: `/v3/groups/90ece6014584498485c079a870f3ffdf` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:40.814489 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e60463-23a9-4c45-ab28-77782a49285b None admin] RBAC: Authorizing `identity:get_group(group_id=90ece6014584498485c079a870f3ffdf)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:40.815946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e60463-23a9-4c45-ab28-77782a49285b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:40.818149 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 195/379] 10.222.0.100 () {56 vars in 1185 bytes} [Tue Aug 2 08:57:40 2022] GET /identity/v3/groups/90ece6014584498485c079a870f3ffdf => generated 230 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:40.819080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e60463-23a9-4c45-ab28-77782a49285b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:40.848296 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:40.939177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f24f2-529d-4961-9455-1d57cce2f97b None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:40.945884 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e60463-23a9-4c45-ab28-77782a49285b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:40.965146 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:40.981368 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22046c5a-017c-4a23-9e74-2c4f0e33bff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:40.982336 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22046c5a-017c-4a23-9e74-2c4f0e33bff8 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:40.982825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22046c5a-017c-4a23-9e74-2c4f0e33bff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:40.983301 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22046c5a-017c-4a23-9e74-2c4f0e33bff8 None admin] PATH_INFO: `/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:40.984656 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22046c5a-017c-4a23-9e74-2c4f0e33bff8 None admin] RBAC: Authorizing `identity:get_group(group_id=f3b15961a0154a2abd8e3ba87e5b37b2)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:40.986512 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22046c5a-017c-4a23-9e74-2c4f0e33bff8 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:40.988403 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 185/380] 10.222.0.100 () {56 vars in 1185 bytes} [Tue Aug 2 08:57:40 2022] GET /identity/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2 => generated 227 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:40.989337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22046c5a-017c-4a23-9e74-2c4f0e33bff8 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:40.991514 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58026956-41af-4c1b-b721-88771648631d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:40.992253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58026956-41af-4c1b-b721-88771648631d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:40.992879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58026956-41af-4c1b-b721-88771648631d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:40.993261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58026956-41af-4c1b-b721-88771648631d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:41.007545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58026956-41af-4c1b-b721-88771648631d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:41.015061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-58026956-41af-4c1b-b721-88771648631d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:41.015061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:41.015061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:41.015061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:41.015061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:41.015061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:41.015061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:41.015061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:41.015061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:41.015061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:41.015061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:41.015061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:41.015061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:41.015061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:41.015061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:41.015061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:41.015951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:41.015951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:41.015951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:41.015951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:41.015951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:41.015951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:41.015951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:41.015951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:41.015951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:41.015951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:41.015951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:41.015951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:41.015951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:41.015951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:41.015951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:41.015951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:41.016763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:41.016763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:41.016763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:41.016763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:41.016763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:41.016763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:41.016763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:41.016763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:41.016763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:41.016763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:41.016763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:41.016763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:41.016763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 196/381] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:41.017472 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58026956-41af-4c1b-b721-88771648631d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:41.114255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22046c5a-017c-4a23-9e74-2c4f0e33bff8 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:41.129772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:41.152701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58026956-41af-4c1b-b721-88771648631d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:41.156327 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8886c27-f45e-49d8-9410-f8ba530302f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:41.157579 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8886c27-f45e-49d8-9410-f8ba530302f9 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:41.158110 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8886c27-f45e-49d8-9410-f8ba530302f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:41.158589 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8886c27-f45e-49d8-9410-f8ba530302f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:41.159748 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8886c27-f45e-49d8-9410-f8ba530302f9 None admin] RBAC: Adding query filter params (group.id=90ece6014584498485c079a870f3ffdf, role.id=7cbff51867a7475d837afd51f6c12f11, scope.project.id=1b02845ab43a49ada6db1e8f52cf0026) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:41.160271 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8886c27-f45e-49d8-9410-f8ba530302f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:41.162042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8886c27-f45e-49d8-9410-f8ba530302f9 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:41.169499 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 186/382] 10.222.0.100 () {56 vars in 1381 bytes} [Tue Aug 2 08:57:40 2022] GET /identity/v3/role_assignments?group.id=90ece6014584498485c079a870f3ffdf&scope.project.id=1b02845ab43a49ada6db1e8f52cf0026&role.id=7cbff51867a7475d837afd51f6c12f11 => generated 261 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:41.170536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8886c27-f45e-49d8-9410-f8ba530302f9 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:41.191033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0262b1b-4cb0-4dca-9d09-b0bcf8694a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:41.191578 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0262b1b-4cb0-4dca-9d09-b0bcf8694a08 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:41.191860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0262b1b-4cb0-4dca-9d09-b0bcf8694a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:41.192132 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0262b1b-4cb0-4dca-9d09-b0bcf8694a08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:41.192924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0262b1b-4cb0-4dca-9d09-b0bcf8694a08 None admin] RBAC: Adding query filter params (group.id=f3b15961a0154a2abd8e3ba87e5b37b2, role.id=bb986e49c38c40bab629ff32371b9358, scope.project.id=51e776c25b1649278e436bb36fe083f4) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:41.193264 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0262b1b-4cb0-4dca-9d09-b0bcf8694a08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:41.199411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0262b1b-4cb0-4dca-9d09-b0bcf8694a08 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:41.205134 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 197/383] 10.222.0.100 () {56 vars in 1381 bytes} [Tue Aug 2 08:57:41 2022] GET /identity/v3/role_assignments?group.id=f3b15961a0154a2abd8e3ba87e5b37b2&scope.project.id=51e776c25b1649278e436bb36fe083f4&role.id=bb986e49c38c40bab629ff32371b9358 => generated 261 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:41.281500 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:41.344614 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8886c27-f45e-49d8-9410-f8ba530302f9 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:41.386161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972438f0-572e-4ee4-a788-a0d0a4dddff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:41.387175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972438f0-572e-4ee4-a788-a0d0a4dddff0 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:41.387663 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972438f0-572e-4ee4-a788-a0d0a4dddff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:41.388139 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972438f0-572e-4ee4-a788-a0d0a4dddff0 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:41.389101 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972438f0-572e-4ee4-a788-a0d0a4dddff0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:41.389674 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972438f0-572e-4ee4-a788-a0d0a4dddff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:41.391290 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972438f0-572e-4ee4-a788-a0d0a4dddff0 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:41.403041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 187/384] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:41 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:41.409232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0262b1b-4cb0-4dca-9d09-b0bcf8694a08 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:41.540236 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0262b1b-4cb0-4dca-9d09-b0bcf8694a08 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:41.580128 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22276bfb-c209-48f8-bcbb-0ff309af32c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:41.581177 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22276bfb-c209-48f8-bcbb-0ff309af32c5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:41.581669 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22276bfb-c209-48f8-bcbb-0ff309af32c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:41.582168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22276bfb-c209-48f8-bcbb-0ff309af32c5 None admin] PATH_INFO: `/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:41.583567 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22276bfb-c209-48f8-bcbb-0ff309af32c5 None admin] RBAC: Authorizing `identity:get_group(group_id=f3b15961a0154a2abd8e3ba87e5b37b2)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:41.585326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22276bfb-c209-48f8-bcbb-0ff309af32c5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:41.587481 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 198/385] 10.222.0.100 () {56 vars in 1185 bytes} [Tue Aug 2 08:57:41 2022] GET /identity/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2 => generated 227 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:41.594264 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972438f0-572e-4ee4-a788-a0d0a4dddff0 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:41.720164 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972438f0-572e-4ee4-a788-a0d0a4dddff0 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:41.742684 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:41.758062 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434bbc10-9050-4a6c-af11-82e9fb9f11c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:41.758712 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bbc10-9050-4a6c-af11-82e9fb9f11c1 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:41.759002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bbc10-9050-4a6c-af11-82e9fb9f11c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:41.759284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bbc10-9050-4a6c-af11-82e9fb9f11c1 None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:41.761178 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434bbc10-9050-4a6c-af11-82e9fb9f11c1 None admin] RBAC: Authorizing `identity:get_project(project_id=a78316c83e17485ebfd6d3c10ec19d8e)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:41.762816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434bbc10-9050-4a6c-af11-82e9fb9f11c1 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:41.764500 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 188/386] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:41 2022] GET /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e => generated 299 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:41.770461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22276bfb-c209-48f8-bcbb-0ff309af32c5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:41.793071 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:41.897139 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd5553-29bb-4bfc-ba03-1f4c2ef035a1 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:41.897139 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd5553-29bb-4bfc-ba03-1f4c2ef035a1 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:41.897139 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd5553-29bb-4bfc-ba03-1f4c2ef035a1 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:41.897469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 189/387] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:41.898291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22276bfb-c209-48f8-bcbb-0ff309af32c5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:41.910647 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc379f9-5f8d-4092-8a19-e279bca7434a None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:41.910940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc379f9-5f8d-4092-8a19-e279bca7434a None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:41.911217 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc379f9-5f8d-4092-8a19-e279bca7434a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:41.934100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef738e-b1d9-41a5-9ac9-8765f9f70c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:41.934791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef738e-b1d9-41a5-9ac9-8765f9f70c88 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:41.935079 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef738e-b1d9-41a5-9ac9-8765f9f70c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:41.935360 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef738e-b1d9-41a5-9ac9-8765f9f70c88 None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e/groups/f3b15961a0154a2abd8e3ba87e5b37b2/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:41.946118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef738e-b1d9-41a5-9ac9-8765f9f70c88 None admin] RBAC: Authorizing `identity:create_grant(project_id=a78316c83e17485ebfd6d3c10ec19d8e, group_id=f3b15961a0154a2abd8e3ba87e5b37b2, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:41.948173 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef738e-b1d9-41a5-9ac9-8765f9f70c88 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:41.959283 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-5cc379f9-5f8d-4092-8a19-e279bca7434a None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:41.960622 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-5cc379f9-5f8d-4092-8a19-e279bca7434a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:41.973367 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 199/388] 10.222.0.100 () {58 vars in 1447 bytes} [Tue Aug 2 08:57:41 2022] PUT /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e/groups/f3b15961a0154a2abd8e3ba87e5b37b2/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:57:41.983644 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134341f-c03e-4496-8895-f7e9baa863ba None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:41.984279 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134341f-c03e-4496-8895-f7e9baa863ba None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:41.984829 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134341f-c03e-4496-8895-f7e9baa863ba None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:41.986612 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 200/389] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:41 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:41.996075 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856aaf3-c12a-4d4f-a7f3-2553322f96a2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:41.996714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856aaf3-c12a-4d4f-a7f3-2553322f96a2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:41.997243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856aaf3-c12a-4d4f-a7f3-2553322f96a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:42.035855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3856aaf3-c12a-4d4f-a7f3-2553322f96a2 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:42.035855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3856aaf3-c12a-4d4f-a7f3-2553322f96a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:42.123231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 190/390] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:57:42.144273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc379f9-5f8d-4092-8a19-e279bca7434a None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:42.178606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 201/391] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 02 08:57:42.215043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856aaf3-c12a-4d4f-a7f3-2553322f96a2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:42.226244 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:42.277937 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc379f9-5f8d-4092-8a19-e279bca7434a None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:42.292696 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:42.317951 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04021d4-67bb-4c25-ad1a-4a0098597a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:42.318953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04021d4-67bb-4c25-ad1a-4a0098597a1a None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:42.319447 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04021d4-67bb-4c25-ad1a-4a0098597a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:42.320267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04021d4-67bb-4c25-ad1a-4a0098597a1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:42.330822 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856aaf3-c12a-4d4f-a7f3-2553322f96a2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:42.333736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04021d4-67bb-4c25-ad1a-4a0098597a1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:42.339206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-c04021d4-67bb-4c25-ad1a-4a0098597a1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:42.339206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:42.339206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:42.339206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:42.339206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:42.339206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:42.339206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:42.339206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:42.339206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:42.339206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:42.339206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:42.339206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:42.339206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:42.339206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:42.339206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:42.339206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:42.340231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:42.340231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:42.340231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:42.340231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:42.340231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:42.340231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:42.340231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:42.340231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:42.340231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:42.340231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:42.340231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:42.340231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:42.340231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:42.340231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:42.340231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:42.340231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:42.341082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:42.341082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:42.341082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:42.341082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:42.341082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:42.341082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:42.341082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:42.341082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:42.341082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:42.341082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:42.341082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:42.341082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:42.343861 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 191/392] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:42.350159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04021d4-67bb-4c25-ad1a-4a0098597a1a None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:42.372877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230d493-1a5a-4eaa-95ec-f83f7f8bd9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:42.373810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230d493-1a5a-4eaa-95ec-f83f7f8bd9fc None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:42.374168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230d493-1a5a-4eaa-95ec-f83f7f8bd9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:42.374541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230d493-1a5a-4eaa-95ec-f83f7f8bd9fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:42.386930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c230d493-1a5a-4eaa-95ec-f83f7f8bd9fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:42.392638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-c230d493-1a5a-4eaa-95ec-f83f7f8bd9fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:42.392638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:42.392638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:42.392638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:42.392638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:42.392638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:42.392638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:42.392638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:42.392638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:42.392638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:42.392638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:42.392638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:42.392638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:42.392638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:42.392638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:42.392638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:42.393719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:42.393719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:42.393719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:42.393719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:42.393719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:42.393719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:42.393719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:42.393719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:42.393719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:42.393719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:42.393719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:42.393719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:42.393719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:42.393719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:42.393719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:42.393719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:42.394531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:42.394531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:42.394531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:42.394531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:42.394531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:42.394531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:42.394531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:42.394531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:42.394531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:42.394531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:42.394531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:42.394531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:42.398689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 202/393] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:57:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:57:42.399864 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230d493-1a5a-4eaa-95ec-f83f7f8bd9fc None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:42.466004 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04021d4-67bb-4c25-ad1a-4a0098597a1a None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:42.474083 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:42.503510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40697849-128f-483e-8e7a-b5cbdef920ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:42.504278 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40697849-128f-483e-8e7a-b5cbdef920ff None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:42.504667 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40697849-128f-483e-8e7a-b5cbdef920ff None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:42.505048 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40697849-128f-483e-8e7a-b5cbdef920ff None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:42.505903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40697849-128f-483e-8e7a-b5cbdef920ff None admin] RBAC: Adding query filter params (name=member) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:42.506323 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40697849-128f-483e-8e7a-b5cbdef920ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:42.507788 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40697849-128f-483e-8e7a-b5cbdef920ff None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:42.513642 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230d493-1a5a-4eaa-95ec-f83f7f8bd9fc None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:42.521145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 192/394] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:42 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:42.527186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40697849-128f-483e-8e7a-b5cbdef920ff None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:42.590345 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3802076b-4d84-404b-83e2-8957e4afd824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:42.591528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802076b-4d84-404b-83e2-8957e4afd824 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:42.593088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802076b-4d84-404b-83e2-8957e4afd824 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:42.593632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802076b-4d84-404b-83e2-8957e4afd824 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:42.594672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3802076b-4d84-404b-83e2-8957e4afd824 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:42.595897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3802076b-4d84-404b-83e2-8957e4afd824 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:42.600236 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3802076b-4d84-404b-83e2-8957e4afd824 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:42.600364 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:42.623162 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 203/395] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:57:42 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:42.625082 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3802076b-4d84-404b-83e2-8957e4afd824 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:42.731496 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3802076b-4d84-404b-83e2-8957e4afd824 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:42.736452 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40697849-128f-483e-8e7a-b5cbdef920ff None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:42.761493 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:42.770602 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956253cc-c34e-460b-bb5b-48255df1dc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:42.771680 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956253cc-c34e-460b-bb5b-48255df1dc46 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:42.772174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956253cc-c34e-460b-bb5b-48255df1dc46 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:42.772670 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956253cc-c34e-460b-bb5b-48255df1dc46 None admin] PATH_INFO: `/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:42.774364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956253cc-c34e-460b-bb5b-48255df1dc46 None admin] RBAC: Authorizing `identity:get_group(group_id=f3b15961a0154a2abd8e3ba87e5b37b2)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:42.776133 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956253cc-c34e-460b-bb5b-48255df1dc46 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:42.781220 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 204/396] 10.222.0.100 () {56 vars in 1185 bytes} [Tue Aug 2 08:57:42 2022] GET /identity/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2 => generated 227 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:42.784491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956253cc-c34e-460b-bb5b-48255df1dc46 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:42.814755 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7c81a-5045-4180-9c19-9750500c2fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:42.815424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7c81a-5045-4180-9c19-9750500c2fbf None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:42.815708 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7c81a-5045-4180-9c19-9750500c2fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:42.815986 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7c81a-5045-4180-9c19-9750500c2fbf None admin] PATH_INFO: `/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:42.817177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b7c81a-5045-4180-9c19-9750500c2fbf None admin] RBAC: Authorizing `identity:get_group(group_id=f3b15961a0154a2abd8e3ba87e5b37b2)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:42.818626 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b7c81a-5045-4180-9c19-9750500c2fbf None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:42.824876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 193/397] 10.222.0.100 () {56 vars in 1185 bytes} [Tue Aug 2 08:57:42 2022] GET /identity/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2 => generated 227 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:42.834535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7c81a-5045-4180-9c19-9750500c2fbf None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:42.901720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956253cc-c34e-460b-bb5b-48255df1dc46 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:42.938880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb387fac-8b9b-4ee0-bba7-f799bcb436db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:42.939852 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb387fac-8b9b-4ee0-bba7-f799bcb436db None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:42.940379 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb387fac-8b9b-4ee0-bba7-f799bcb436db None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:42.940862 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb387fac-8b9b-4ee0-bba7-f799bcb436db None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:42.942351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb387fac-8b9b-4ee0-bba7-f799bcb436db None admin] RBAC: Authorizing `identity:get_project(project_id=a78316c83e17485ebfd6d3c10ec19d8e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:42.944100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb387fac-8b9b-4ee0-bba7-f799bcb436db None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:42.949263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 205/398] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:42 2022] GET /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e => generated 299 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:42.952852 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb387fac-8b9b-4ee0-bba7-f799bcb436db None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:43.033650 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:43.035940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7c81a-5045-4180-9c19-9750500c2fbf None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:43.038560 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:43.071134 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb387fac-8b9b-4ee0-bba7-f799bcb436db None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:43.109176 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf67b09-c67d-41eb-812e-6768d065af60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:43.110280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf67b09-c67d-41eb-812e-6768d065af60 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:43.110810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf67b09-c67d-41eb-812e-6768d065af60 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:43.111303 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf67b09-c67d-41eb-812e-6768d065af60 None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:43.113568 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf67b09-c67d-41eb-812e-6768d065af60 None admin] RBAC: Authorizing `identity:get_project(project_id=51e776c25b1649278e436bb36fe083f4)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:43.120748 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf67b09-c67d-41eb-812e-6768d065af60 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:43.124248 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 194/399] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:42 2022] GET /identity/v3/projects/51e776c25b1649278e436bb36fe083f4 => generated 303 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:43.134570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf67b09-c67d-41eb-812e-6768d065af60 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:43.147961 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225987c5-aeb0-4ec0-a1d8-491a96805e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:43.148929 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225987c5-aeb0-4ec0-a1d8-491a96805e6e None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:43.150273 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225987c5-aeb0-4ec0-a1d8-491a96805e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:43.151336 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225987c5-aeb0-4ec0-a1d8-491a96805e6e None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e/groups/f3b15961a0154a2abd8e3ba87e5b37b2/roles/bb986e49c38c40bab629ff32371b9358` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:43.156062 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225987c5-aeb0-4ec0-a1d8-491a96805e6e None admin] RBAC: Authorizing `identity:create_grant(project_id=a78316c83e17485ebfd6d3c10ec19d8e, group_id=f3b15961a0154a2abd8e3ba87e5b37b2, role_id=bb986e49c38c40bab629ff32371b9358)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:43.159437 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225987c5-aeb0-4ec0-a1d8-491a96805e6e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:43.197214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 206/400] 10.222.0.100 () {58 vars in 1447 bytes} [Tue Aug 2 08:57:42 2022] PUT /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e/groups/f3b15961a0154a2abd8e3ba87e5b37b2/roles/bb986e49c38c40bab629ff32371b9358 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:57:43.282508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf67b09-c67d-41eb-812e-6768d065af60 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:43.324873 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec985c7-5b31-478d-8341-58326ebed0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:43.325924 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec985c7-5b31-478d-8341-58326ebed0ea None admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:43.326495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec985c7-5b31-478d-8341-58326ebed0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:43.326959 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec985c7-5b31-478d-8341-58326ebed0ea None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4/groups/f3b15961a0154a2abd8e3ba87e5b37b2/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:43.329156 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec985c7-5b31-478d-8341-58326ebed0ea None admin] RBAC: Authorizing `identity:create_grant(project_id=51e776c25b1649278e436bb36fe083f4, group_id=f3b15961a0154a2abd8e3ba87e5b37b2, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:43.330827 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec985c7-5b31-478d-8341-58326ebed0ea None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:43.349251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 195/401] 10.222.0.100 () {58 vars in 1447 bytes} [Tue Aug 2 08:57:43 2022] PUT /identity/v3/projects/51e776c25b1649278e436bb36fe083f4/groups/f3b15961a0154a2abd8e3ba87e5b37b2/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:57:43.597156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ea673-25f5-4adb-8392-22fc788ff98f None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:43.597690 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ea673-25f5-4adb-8392-22fc788ff98f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:43.597776 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ea673-25f5-4adb-8392-22fc788ff98f None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:43.599527 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 207/402] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:43.604612 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4ff94-1828-4c4e-a06a-aeea9a6d307e None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:43.604916 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4ff94-1828-4c4e-a06a-aeea9a6d307e None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:43.605231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4ff94-1828-4c4e-a06a-aeea9a6d307e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:43.630495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-81f4ff94-1828-4c4e-a06a-aeea9a6d307e None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:43.632044 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-81f4ff94-1828-4c4e-a06a-aeea9a6d307e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:43.667790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:43.707430 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475258e-20b5-43d4-bdff-e8270543b0cb None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:43.707775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475258e-20b5-43d4-bdff-e8270543b0cb None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:43.707995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475258e-20b5-43d4-bdff-e8270543b0cb None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:43.709407 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 208/403] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:43.715704 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d34d3d-81e6-4e59-8e9d-287085763599 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:43.716024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d34d3d-81e6-4e59-8e9d-287085763599 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:43.716278 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d34d3d-81e6-4e59-8e9d-287085763599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:43.725785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 196/404] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:43 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:43.742767 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-58d34d3d-81e6-4e59-8e9d-287085763599 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:43.744030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-58d34d3d-81e6-4e59-8e9d-287085763599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:43.754003 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4ff94-1828-4c4e-a06a-aeea9a6d307e None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:43.845299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 209/405] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:43 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:43.912871 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4ff94-1828-4c4e-a06a-aeea9a6d307e None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:43.923459 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d34d3d-81e6-4e59-8e9d-287085763599 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:43.975320 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b1a5c-68ce-4667-9f53-b0fa4b1b17d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:43.975976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b1a5c-68ce-4667-9f53-b0fa4b1b17d6 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:43.976256 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b1a5c-68ce-4667-9f53-b0fa4b1b17d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:43.976531 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b1a5c-68ce-4667-9f53-b0fa4b1b17d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:44.001796 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:44.009137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b1a5c-68ce-4667-9f53-b0fa4b1b17d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:44.010776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-1b6b1a5c-68ce-4667-9f53-b0fa4b1b17d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:44.010776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:44.010776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:44.010776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:44.010776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:44.010776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:44.010776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:44.010776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:44.010776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:44.010776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:44.010776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:44.010776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:44.010776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:44.010776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:44.010776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:44.010776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:44.011736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:44.011736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:44.011736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:44.011736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:44.011736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:44.011736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:44.011736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:44.011736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:44.011736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:44.011736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:44.011736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:44.011736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:44.011736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:44.011736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:44.011736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:44.011736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:44.012607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:44.012607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:44.012607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:44.012607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:44.012607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:44.012607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:44.012607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:44.012607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:44.012607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:44.012607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:44.012607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:44.012607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:44.012607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 197/406] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:57:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:44.017903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b1a5c-68ce-4667-9f53-b0fa4b1b17d6 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:44.053641 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:44.074697 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d34d3d-81e6-4e59-8e9d-287085763599 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:44.146381 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8b57c-76f7-4671-806e-396dcc2a0fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:44.147311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8b57c-76f7-4671-806e-396dcc2a0fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:44.147741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8b57c-76f7-4671-806e-396dcc2a0fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:44.148177 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8b57c-76f7-4671-806e-396dcc2a0fa8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:44.171216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8b57c-76f7-4671-806e-396dcc2a0fa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:44.176720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-4aa8b57c-76f7-4671-806e-396dcc2a0fa8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:57:44.176720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:44.176720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:44.176720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:44.176720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:44.176720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:44.176720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:44.176720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:44.176720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:44.176720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:44.176720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:44.176720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:44.176720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:44.176720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:44.176720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:44.176720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:44.177820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:44.177820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:44.177820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:44.177820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:44.177820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:44.177820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:44.177820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:44.177820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:44.177820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:44.177820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:44.177820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:44.177820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:44.177820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:44.177820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:44.177820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:44.177820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:44.178898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:44.178898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:44.178898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:44.178898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:44.178898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:44.178898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:44.178898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:44.178898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:44.178898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:44.178898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:44.178898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:57:44.178898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:44.187566 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 210/407] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:57:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:57:44.193501 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8b57c-76f7-4671-806e-396dcc2a0fa8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:44.216506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b1a5c-68ce-4667-9f53-b0fa4b1b17d6 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:44.287679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46871c47-215a-4feb-ab02-9a2299188ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:44.289009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46871c47-215a-4feb-ab02-9a2299188ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:44.289490 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46871c47-215a-4feb-ab02-9a2299188ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:44.289913 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46871c47-215a-4feb-ab02-9a2299188ce9 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:44.290787 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46871c47-215a-4feb-ab02-9a2299188ce9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:44.291776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46871c47-215a-4feb-ab02-9a2299188ce9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:44.293354 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46871c47-215a-4feb-ab02-9a2299188ce9 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:44.313636 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 198/408] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:57:44 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:44.317926 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46871c47-215a-4feb-ab02-9a2299188ce9 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:44.341021 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8b57c-76f7-4671-806e-396dcc2a0fa8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:44.373490 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:44.410277 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c0f56-7b65-4bc6-820d-c0cb6b5dc53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:44.411299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c0f56-7b65-4bc6-820d-c0cb6b5dc53e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:44.412565 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c0f56-7b65-4bc6-820d-c0cb6b5dc53e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:44.413739 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c0f56-7b65-4bc6-820d-c0cb6b5dc53e None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:44.414760 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c0f56-7b65-4bc6-820d-c0cb6b5dc53e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:44.416780 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c0f56-7b65-4bc6-820d-c0cb6b5dc53e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:44.418329 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c0f56-7b65-4bc6-820d-c0cb6b5dc53e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:44.432473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 211/409] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:44 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:44.438892 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c0f56-7b65-4bc6-820d-c0cb6b5dc53e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:44.484549 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:44.507656 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46871c47-215a-4feb-ab02-9a2299188ce9 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:44.555190 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cdeea-50a5-4d43-9386-cb90f64930ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:44.556142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cdeea-50a5-4d43-9386-cb90f64930ea None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:44.557273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cdeea-50a5-4d43-9386-cb90f64930ea None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:44.557846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cdeea-50a5-4d43-9386-cb90f64930ea None admin] PATH_INFO: `/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:44.565826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26cdeea-50a5-4d43-9386-cb90f64930ea None admin] RBAC: Authorizing `identity:get_group(group_id=f3b15961a0154a2abd8e3ba87e5b37b2)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:44.567519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26cdeea-50a5-4d43-9386-cb90f64930ea None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:44.569969 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 199/410] 10.222.0.100 () {56 vars in 1185 bytes} [Tue Aug 2 08:57:44 2022] GET /identity/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2 => generated 227 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:44.576082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cdeea-50a5-4d43-9386-cb90f64930ea None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:44.596958 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c0f56-7b65-4bc6-820d-c0cb6b5dc53e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:44.642443 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658ed6e-cd4f-42f2-8b68-1e3bec07047b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:44.643851 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658ed6e-cd4f-42f2-8b68-1e3bec07047b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:44.644381 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658ed6e-cd4f-42f2-8b68-1e3bec07047b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:44.644971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658ed6e-cd4f-42f2-8b68-1e3bec07047b None admin] PATH_INFO: `/v3/groups/90ece6014584498485c079a870f3ffdf` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:44.647179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658ed6e-cd4f-42f2-8b68-1e3bec07047b None admin] RBAC: Authorizing `identity:get_group(group_id=90ece6014584498485c079a870f3ffdf)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:44.653583 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658ed6e-cd4f-42f2-8b68-1e3bec07047b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:44.655889 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 212/411] 10.222.0.100 () {56 vars in 1185 bytes} [Tue Aug 2 08:57:44 2022] GET /identity/v3/groups/90ece6014584498485c079a870f3ffdf => generated 230 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:44.660921 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658ed6e-cd4f-42f2-8b68-1e3bec07047b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:44.737541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:44.744041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cdeea-50a5-4d43-9386-cb90f64930ea None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:44.791650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c83f29-77bc-4633-a116-5ab45823c625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:44.792771 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c83f29-77bc-4633-a116-5ab45823c625 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:44.793231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c83f29-77bc-4633-a116-5ab45823c625 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:44.793653 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c83f29-77bc-4633-a116-5ab45823c625 None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:44.793805 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658ed6e-cd4f-42f2-8b68-1e3bec07047b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:44.795121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c83f29-77bc-4633-a116-5ab45823c625 None admin] RBAC: Authorizing `identity:get_project(project_id=51e776c25b1649278e436bb36fe083f4)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:44.797923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c83f29-77bc-4633-a116-5ab45823c625 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:44.804657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 200/412] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:44 2022] GET /identity/v3/projects/51e776c25b1649278e436bb36fe083f4 => generated 303 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:44.805471 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462640b-d485-4f0a-ae45-9347bff39fd3 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:44.805892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462640b-d485-4f0a-ae45-9347bff39fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:44.805995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462640b-d485-4f0a-ae45-9347bff39fd3 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:44.807229 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 201/413] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:44.808562 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462640b-d485-4f0a-ae45-9347bff39fd3 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:44.832735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:44.847790 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff36c19-710e-4788-94e0-b5778204fd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:44.849274 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff36c19-710e-4788-94e0-b5778204fd83 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:44.849573 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff36c19-710e-4788-94e0-b5778204fd83 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:44.849860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff36c19-710e-4788-94e0-b5778204fd83 None admin] PATH_INFO: `/v3/projects/1b02845ab43a49ada6db1e8f52cf0026` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:44.851589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff36c19-710e-4788-94e0-b5778204fd83 None admin] RBAC: Authorizing `identity:get_project(project_id=1b02845ab43a49ada6db1e8f52cf0026)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:44.853317 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff36c19-710e-4788-94e0-b5778204fd83 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:44.860139 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 213/414] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:44 2022] GET /identity/v3/projects/1b02845ab43a49ada6db1e8f52cf0026 => generated 345 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:44.860139 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880a37a-8c8f-4909-b629-c426d5b3bef3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:44.860139 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880a37a-8c8f-4909-b629-c426d5b3bef3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:44.860139 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880a37a-8c8f-4909-b629-c426d5b3bef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:44.878322 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-2880a37a-8c8f-4909-b629-c426d5b3bef3 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:44.879506 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2880a37a-8c8f-4909-b629-c426d5b3bef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:44.948020 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462640b-d485-4f0a-ae45-9347bff39fd3 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:44.969132 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 214/415] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:44 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:44.969132 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880a37a-8c8f-4909-b629-c426d5b3bef3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:45.005637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:45.007258 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dceabd2-aa46-4e6a-8c0f-a5f80ab3febe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:45.008249 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dceabd2-aa46-4e6a-8c0f-a5f80ab3febe None admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:45.008753 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dceabd2-aa46-4e6a-8c0f-a5f80ab3febe None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:45.017347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dceabd2-aa46-4e6a-8c0f-a5f80ab3febe None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4/groups/f3b15961a0154a2abd8e3ba87e5b37b2/roles/bb986e49c38c40bab629ff32371b9358` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:45.020190 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dceabd2-aa46-4e6a-8c0f-a5f80ab3febe None admin] RBAC: Authorizing `identity:create_grant(project_id=51e776c25b1649278e436bb36fe083f4, group_id=f3b15961a0154a2abd8e3ba87e5b37b2, role_id=bb986e49c38c40bab629ff32371b9358)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:45.022702 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dceabd2-aa46-4e6a-8c0f-a5f80ab3febe None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:45.053036 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 202/416] 10.222.0.100 () {58 vars in 1447 bytes} [Tue Aug 2 08:57:44 2022] PUT /identity/v3/projects/51e776c25b1649278e436bb36fe083f4/groups/f3b15961a0154a2abd8e3ba87e5b37b2/roles/bb986e49c38c40bab629ff32371b9358 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:57:45.053036 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dceabd2-aa46-4e6a-8c0f-a5f80ab3febe None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:45.094962 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880a37a-8c8f-4909-b629-c426d5b3bef3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:45.176241 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0cb927-ec48-4675-992d-4e10bc626b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:45.176933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0cb927-ec48-4675-992d-4e10bc626b95 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:45.177255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0cb927-ec48-4675-992d-4e10bc626b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:45.177539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0cb927-ec48-4675-992d-4e10bc626b95 None admin] PATH_INFO: `/v3/projects/1b02845ab43a49ada6db1e8f52cf0026/groups/90ece6014584498485c079a870f3ffdf/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:45.184179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0cb927-ec48-4675-992d-4e10bc626b95 None admin] RBAC: Authorizing `identity:create_grant(project_id=1b02845ab43a49ada6db1e8f52cf0026, group_id=90ece6014584498485c079a870f3ffdf, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:45.184179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0cb927-ec48-4675-992d-4e10bc626b95 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:45.203022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 215/417] 10.222.0.100 () {58 vars in 1447 bytes} [Tue Aug 2 08:57:44 2022] PUT /identity/v3/projects/1b02845ab43a49ada6db1e8f52cf0026/groups/90ece6014584498485c079a870f3ffdf/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:57:45.220100 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:45.241490 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dceabd2-aa46-4e6a-8c0f-a5f80ab3febe None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:45.286445 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b441b11-97c1-4eac-9a4f-1ea2ed1a8724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:45.287012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b441b11-97c1-4eac-9a4f-1ea2ed1a8724 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:45.287296 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b441b11-97c1-4eac-9a4f-1ea2ed1a8724 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:45.287570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b441b11-97c1-4eac-9a4f-1ea2ed1a8724 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:45.300797 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b441b11-97c1-4eac-9a4f-1ea2ed1a8724 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:45.306150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-2b441b11-97c1-4eac-9a4f-1ea2ed1a8724 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:45.306150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:45.306150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:45.306150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:45.306150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:45.306150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:45.306150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:45.306150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:45.306150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:45.306150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:45.306150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:45.306150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:45.306150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:45.306150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:45.306150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:45.306150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:45.307128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:45.307128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:45.307128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:45.307128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:45.307128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:45.307128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:45.307128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:45.307128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:45.307128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:45.307128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:45.307128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:45.307128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:45.307128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:45.307128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:45.307128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:45.307128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:45.307940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:45.307940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:45.307940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:45.307940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:45.307940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:45.307940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:45.307940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:45.307940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:45.307940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:45.307940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:45.307940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:45.307940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:45.309074 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 203/418] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:45.316548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0cb927-ec48-4675-992d-4e10bc626b95 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:45.431773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0cb927-ec48-4675-992d-4e10bc626b95 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:45.433612 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:45.470184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64affe1-a67c-4ff2-be38-8ee8271d5953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:45.470819 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64affe1-a67c-4ff2-be38-8ee8271d5953 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:45.471107 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64affe1-a67c-4ff2-be38-8ee8271d5953 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:45.471385 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64affe1-a67c-4ff2-be38-8ee8271d5953 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:45.472108 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64affe1-a67c-4ff2-be38-8ee8271d5953 None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:45.472416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64affe1-a67c-4ff2-be38-8ee8271d5953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:45.473812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64affe1-a67c-4ff2-be38-8ee8271d5953 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:45.483724 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 216/419] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:45 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:45.489808 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b441b11-97c1-4eac-9a4f-1ea2ed1a8724 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:45.607208 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b441b11-97c1-4eac-9a4f-1ea2ed1a8724 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:45.642819 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527a4f3-1f23-440f-b981-4a20528bbd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:45.643443 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527a4f3-1f23-440f-b981-4a20528bbd46 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:45.643724 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527a4f3-1f23-440f-b981-4a20528bbd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:45.644000 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527a4f3-1f23-440f-b981-4a20528bbd46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:45.653538 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:45.735690 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527a4f3-1f23-440f-b981-4a20528bbd46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:45.737176 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527a4f3-1f23-440f-b981-4a20528bbd46 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:45.765055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe769fc-ffff-4d28-bfeb-8b1456f6d1a3 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:45.765603 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe769fc-ffff-4d28-bfeb-8b1456f6d1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:45.765680 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe769fc-ffff-4d28-bfeb-8b1456f6d1a3 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:45.767298 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 217/420] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:57:45.772362 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc07d2-37b4-4a5d-adc7-1b09331d10e1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:45.772651 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc07d2-37b4-4a5d-adc7-1b09331d10e1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:45.772929 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc07d2-37b4-4a5d-adc7-1b09331d10e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:45.805747 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-38dc07d2-37b4-4a5d-adc7-1b09331d10e1 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:45.807522 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-38dc07d2-37b4-4a5d-adc7-1b09331d10e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:45.882145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 218/422] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:45 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:57:45.882145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7df8a-6af8-4a0a-a5d2-3d6f27df5b8b None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:45.882145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7df8a-6af8-4a0a-a5d2-3d6f27df5b8b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:45.882145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7df8a-6af8-4a0a-a5d2-3d6f27df5b8b None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:45.882833 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 204/422] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:45 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:45.884863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527a4f3-1f23-440f-b981-4a20528bbd46 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:45.897151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 219/423] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:45 2022] GET /identity => generated 271 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:45.910790 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7df8a-6af8-4a0a-a5d2-3d6f27df5b8b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:45.937198 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:46.015361 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527a4f3-1f23-440f-b981-4a20528bbd46 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:46.039719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7df8a-6af8-4a0a-a5d2-3d6f27df5b8b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:46.051565 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:46.053575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0eb42-33c7-4c04-9889-5decf86e401a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:46.054210 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0eb42-33c7-4c04-9889-5decf86e401a None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:46.054491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0eb42-33c7-4c04-9889-5decf86e401a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:46.054763 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0eb42-33c7-4c04-9889-5decf86e401a None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:46.056102 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b0eb42-33c7-4c04-9889-5decf86e401a None admin] RBAC: Authorizing `identity:get_project(project_id=a78316c83e17485ebfd6d3c10ec19d8e)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:46.057722 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b0eb42-33c7-4c04-9889-5decf86e401a None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:46.061157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 205/424] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:45 2022] GET /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e => generated 299 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:46.061157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019c88a-4473-4123-99d8-ee307b14ce85 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:46.061157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019c88a-4473-4123-99d8-ee307b14ce85 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:46.061434 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019c88a-4473-4123-99d8-ee307b14ce85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:46.079835 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-3019c88a-4473-4123-99d8-ee307b14ce85 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:46.081024 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3019c88a-4473-4123-99d8-ee307b14ce85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:46.091796 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c0ca2-ad91-453b-a8ed-67ea7f540c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:46.092597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c0ca2-ad91-453b-a8ed-67ea7f540c3c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:46.093454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c0ca2-ad91-453b-a8ed-67ea7f540c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:46.093832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c0ca2-ad91-453b-a8ed-67ea7f540c3c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:46.108934 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c0ca2-ad91-453b-a8ed-67ea7f540c3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:46.116658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-300c0ca2-ad91-453b-a8ed-67ea7f540c3c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:46.116658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:46.116658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:46.116658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:46.116658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:46.116658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:46.116658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:46.116658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:46.116658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:46.116658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:46.116658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:46.116658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:46.116658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:46.116658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:46.116658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:46.116658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:46.117772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:46.117772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:46.117772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:46.117772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:46.117772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:46.117772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:46.117772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:46.117772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:46.117772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:46.117772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:46.117772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:46.117772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:46.117772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:46.117772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:46.117772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:46.117772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:46.118935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:46.118935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:46.118935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:46.118935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:46.118935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:46.118935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:46.118935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:46.118935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:46.118935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:46.118935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:46.118935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:46.118935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:46.125132 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 220/425] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:57:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:46.125132 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c0ca2-ad91-453b-a8ed-67ea7f540c3c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:46.165296 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 206/426] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:46 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:46.166279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019c88a-4473-4123-99d8-ee307b14ce85 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:46.223374 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:46.257306 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c0ca2-ad91-453b-a8ed-67ea7f540c3c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:46.294820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf02798-6ef0-44fd-a000-a1e4438def22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:46.295609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf02798-6ef0-44fd-a000-a1e4438def22 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:46.295993 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf02798-6ef0-44fd-a000-a1e4438def22 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:46.296365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf02798-6ef0-44fd-a000-a1e4438def22 None admin] PATH_INFO: `/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:46.297655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf02798-6ef0-44fd-a000-a1e4438def22 None admin] RBAC: Authorizing `identity:get_group(group_id=f3b15961a0154a2abd8e3ba87e5b37b2)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:46.299197 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf02798-6ef0-44fd-a000-a1e4438def22 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:46.301249 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 221/427] 10.222.0.100 () {56 vars in 1185 bytes} [Tue Aug 2 08:57:46 2022] GET /identity/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2 => generated 227 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:46.302219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf02798-6ef0-44fd-a000-a1e4438def22 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:46.309254 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019c88a-4473-4123-99d8-ee307b14ce85 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:46.329769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:46.358308 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f302cdc-6dd5-4be0-baad-7f1655e3f311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:46.358935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f302cdc-6dd5-4be0-baad-7f1655e3f311 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:46.359214 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f302cdc-6dd5-4be0-baad-7f1655e3f311 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:46.359487 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f302cdc-6dd5-4be0-baad-7f1655e3f311 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:46.360200 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f302cdc-6dd5-4be0-baad-7f1655e3f311 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:46.360503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f302cdc-6dd5-4be0-baad-7f1655e3f311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:46.361908 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f302cdc-6dd5-4be0-baad-7f1655e3f311 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:46.370677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 207/428] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:57:46 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:46.371281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f302cdc-6dd5-4be0-baad-7f1655e3f311 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:46.424044 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf02798-6ef0-44fd-a000-a1e4438def22 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:46.465922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4dee9-f3f2-43c0-92a5-ed166e6d27f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:46.466938 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4dee9-f3f2-43c0-92a5-ed166e6d27f9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:46.467413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4dee9-f3f2-43c0-92a5-ed166e6d27f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:46.467974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4dee9-f3f2-43c0-92a5-ed166e6d27f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:46.480043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4dee9-f3f2-43c0-92a5-ed166e6d27f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:46.484678 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:46.486749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-70d4dee9-f3f2-43c0-92a5-ed166e6d27f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:46.486749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:46.486749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:46.486749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:46.486749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:46.486749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:46.486749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:46.486749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:46.486749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:46.486749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:46.486749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:46.486749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:46.486749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:46.486749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:46.486749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:46.486749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:46.487609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:46.487609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:46.487609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:46.487609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:46.487609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:46.487609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:46.487609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:46.487609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:46.487609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:46.487609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:46.487609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:46.487609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:46.487609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:46.487609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:46.487609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:46.487609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:46.488411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:46.488411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:46.488411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:46.488411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:46.488411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:46.488411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:46.488411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:46.488411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:46.488411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:46.488411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:46.488411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:57:46.488411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:46.489006 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f302cdc-6dd5-4be0-baad-7f1655e3f311 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:46.491887 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 222/429] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:46.492860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4dee9-f3f2-43c0-92a5-ed166e6d27f9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:46.532974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd5313-2540-415d-a62c-46a3219e6634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:46.534013 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd5313-2540-415d-a62c-46a3219e6634 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:46.534297 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd5313-2540-415d-a62c-46a3219e6634 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:46.534573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd5313-2540-415d-a62c-46a3219e6634 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:46.535607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd5313-2540-415d-a62c-46a3219e6634 None admin] RBAC: Adding query filter params (group.id=f3b15961a0154a2abd8e3ba87e5b37b2, role.id=f739a9c857cd4050896a6814c8bc0120, scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:46.535921 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd5313-2540-415d-a62c-46a3219e6634 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:46.537660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd5313-2540-415d-a62c-46a3219e6634 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:46.545767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 208/430] 10.222.0.100 () {56 vars in 1381 bytes} [Tue Aug 2 08:57:46 2022] GET /identity/v3/role_assignments?group.id=f3b15961a0154a2abd8e3ba87e5b37b2&scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e&role.id=f739a9c857cd4050896a6814c8bc0120 => generated 613 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:46.546598 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd5313-2540-415d-a62c-46a3219e6634 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:46.597363 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:46.671169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4dee9-f3f2-43c0-92a5-ed166e6d27f9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:46.677974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd5313-2540-415d-a62c-46a3219e6634 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:46.718800 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81becdc-6646-4fd0-9999-daa9ae88b5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:46.719433 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81becdc-6646-4fd0-9999-daa9ae88b5ea None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:46.719778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81becdc-6646-4fd0-9999-daa9ae88b5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:46.720058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81becdc-6646-4fd0-9999-daa9ae88b5ea None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:46.720770 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81becdc-6646-4fd0-9999-daa9ae88b5ea None admin] RBAC: Adding query filter params (name=member) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:46.721082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81becdc-6646-4fd0-9999-daa9ae88b5ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:46.722755 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81becdc-6646-4fd0-9999-daa9ae88b5ea None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:46.733659 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 209/431] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:57:46 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:46.738701 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81becdc-6646-4fd0-9999-daa9ae88b5ea None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:46.751057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe7a9b-3c8f-48c6-9320-cd2b4396bd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:46.751931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe7a9b-3c8f-48c6-9320-cd2b4396bd45 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:46.752216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe7a9b-3c8f-48c6-9320-cd2b4396bd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:46.752494 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe7a9b-3c8f-48c6-9320-cd2b4396bd45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:46.773471 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:46.833549 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe7a9b-3c8f-48c6-9320-cd2b4396bd45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:46.835055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe7a9b-3c8f-48c6-9320-cd2b4396bd45 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:46.871250 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81becdc-6646-4fd0-9999-daa9ae88b5ea None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:46.913613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f2914-d9cf-4cfb-a331-42f7f2873f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:46.914827 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f2914-d9cf-4cfb-a331-42f7f2873f78 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:46.915309 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f2914-d9cf-4cfb-a331-42f7f2873f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:46.915856 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f2914-d9cf-4cfb-a331-42f7f2873f78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:46.963304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:46.984586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 223/432] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:46 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:46.991332 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe7a9b-3c8f-48c6-9320-cd2b4396bd45 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:46.999669 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f2914-d9cf-4cfb-a331-42f7f2873f78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:47.001690 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f2914-d9cf-4cfb-a331-42f7f2873f78 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:47.040036 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:47.138068 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe7a9b-3c8f-48c6-9320-cd2b4396bd45 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:47.142099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 210/433] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:46 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 08:57:47.143053 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f2914-d9cf-4cfb-a331-42f7f2873f78 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:47.184340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e8109-4630-4634-986e-7cb6636d5463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:47.185853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e8109-4630-4634-986e-7cb6636d5463 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:47.185853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e8109-4630-4634-986e-7cb6636d5463 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:47.186131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e8109-4630-4634-986e-7cb6636d5463 None admin] PATH_INFO: `/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:47.189858 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e8109-4630-4634-986e-7cb6636d5463 None admin] RBAC: Authorizing `identity:get_project(project_id=a78316c83e17485ebfd6d3c10ec19d8e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:47.191991 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e8109-4630-4634-986e-7cb6636d5463 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:47.194684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 224/434] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:46 2022] GET /identity/v3/projects/a78316c83e17485ebfd6d3c10ec19d8e => generated 299 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:47.202263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e8109-4630-4634-986e-7cb6636d5463 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:47.271101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:47.283399 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f2914-d9cf-4cfb-a331-42f7f2873f78 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:47.313799 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:47.331862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bf3c3-39ac-4395-a795-483a23472506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:47.332510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bf3c3-39ac-4395-a795-483a23472506 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:47.332758 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e8109-4630-4634-986e-7cb6636d5463 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:47.333040 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bf3c3-39ac-4395-a795-483a23472506 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:47.333040 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bf3c3-39ac-4395-a795-483a23472506 None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:47.334363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bf3c3-39ac-4395-a795-483a23472506 None admin] RBAC: Authorizing `identity:get_project(project_id=51e776c25b1649278e436bb36fe083f4)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:47.337064 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bf3c3-39ac-4395-a795-483a23472506 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:47.339816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 211/435] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:47 2022] GET /identity/v3/projects/51e776c25b1649278e436bb36fe083f4 => generated 303 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:47.345158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bf3c3-39ac-4395-a795-483a23472506 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:47.371552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d7359-1e5b-44e6-a4bc-092d3333cc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:47.372198 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d7359-1e5b-44e6-a4bc-092d3333cc0b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:47.372534 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d7359-1e5b-44e6-a4bc-092d3333cc0b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:47.372806 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d7359-1e5b-44e6-a4bc-092d3333cc0b None admin] PATH_INFO: `/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:47.374001 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d7359-1e5b-44e6-a4bc-092d3333cc0b None admin] RBAC: Authorizing `identity:get_group(group_id=f3b15961a0154a2abd8e3ba87e5b37b2)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:47.375573 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d7359-1e5b-44e6-a4bc-092d3333cc0b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:47.382659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 225/436] 10.222.0.100 () {56 vars in 1185 bytes} [Tue Aug 2 08:57:47 2022] GET /identity/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2 => generated 227 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:47.383816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d7359-1e5b-44e6-a4bc-092d3333cc0b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:47.488365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bf3c3-39ac-4395-a795-483a23472506 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:47.496632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d7359-1e5b-44e6-a4bc-092d3333cc0b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:47.515883 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:47.529000 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a4916-5e59-407a-b959-fb5ca0bd4e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:47.529704 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a4916-5e59-407a-b959-fb5ca0bd4e51 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:47.530003 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a4916-5e59-407a-b959-fb5ca0bd4e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:47.530265 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a4916-5e59-407a-b959-fb5ca0bd4e51 None admin] PATH_INFO: `/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:47.531546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a4916-5e59-407a-b959-fb5ca0bd4e51 None admin] RBAC: Authorizing `identity:get_group(group_id=f3b15961a0154a2abd8e3ba87e5b37b2)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:47.533209 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a4916-5e59-407a-b959-fb5ca0bd4e51 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:47.535305 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 212/437] 10.222.0.100 () {56 vars in 1185 bytes} [Tue Aug 2 08:57:47 2022] GET /identity/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2 => generated 227 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:47.536692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716c66f-2b23-431f-90ea-4d84b952d411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:47.537422 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716c66f-2b23-431f-90ea-4d84b952d411 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:47.537706 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716c66f-2b23-431f-90ea-4d84b952d411 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:47.537982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716c66f-2b23-431f-90ea-4d84b952d411 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:47.538775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716c66f-2b23-431f-90ea-4d84b952d411 None admin] RBAC: Adding query filter params (group.id=f3b15961a0154a2abd8e3ba87e5b37b2, role.id=bb986e49c38c40bab629ff32371b9358, scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:47.539097 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716c66f-2b23-431f-90ea-4d84b952d411 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:47.541690 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716c66f-2b23-431f-90ea-4d84b952d411 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:47.542839 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a4916-5e59-407a-b959-fb5ca0bd4e51 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:47.553418 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 226/438] 10.222.0.100 () {56 vars in 1381 bytes} [Tue Aug 2 08:57:47 2022] GET /identity/v3/role_assignments?group.id=f3b15961a0154a2abd8e3ba87e5b37b2&scope.project.id=a78316c83e17485ebfd6d3c10ec19d8e&role.id=bb986e49c38c40bab629ff32371b9358 => generated 613 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:47.612809 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:47.660198 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a4916-5e59-407a-b959-fb5ca0bd4e51 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:47.681740 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e530f-e19a-405f-8e42-e08c26afe522 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:47.682224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e530f-e19a-405f-8e42-e08c26afe522 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:47.682709 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e530f-e19a-405f-8e42-e08c26afe522 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:47.689593 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 227/439] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:47 2022] GET /identity => generated 271 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:47.694160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b44676-1b0d-4041-adc3-e3ded597480d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:47.694621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b44676-1b0d-4041-adc3-e3ded597480d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:47.695123 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b44676-1b0d-4041-adc3-e3ded597480d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:47.697611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6c66d-01f4-49b8-b70f-df2250618564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:47.698243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6c66d-01f4-49b8-b70f-df2250618564 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:47.698526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6c66d-01f4-49b8-b70f-df2250618564 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:47.698789 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6c66d-01f4-49b8-b70f-df2250618564 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:47.699595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6c66d-01f4-49b8-b70f-df2250618564 None admin] RBAC: Adding query filter params (group.id=f3b15961a0154a2abd8e3ba87e5b37b2, role.id=f739a9c857cd4050896a6814c8bc0120, scope.project.id=51e776c25b1649278e436bb36fe083f4) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:47.699946 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6c66d-01f4-49b8-b70f-df2250618564 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:47.701631 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6c66d-01f4-49b8-b70f-df2250618564 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:47.709369 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 213/440] 10.222.0.100 () {56 vars in 1381 bytes} [Tue Aug 2 08:57:47 2022] GET /identity/v3/role_assignments?group.id=f3b15961a0154a2abd8e3ba87e5b37b2&scope.project.id=51e776c25b1649278e436bb36fe083f4&role.id=f739a9c857cd4050896a6814c8bc0120 => generated 613 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:47.726821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a5b44676-1b0d-4041-adc3-e3ded597480d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:47.729533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a5b44676-1b0d-4041-adc3-e3ded597480d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:47.820715 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 228/441] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:47 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:57:47.844890 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6c66d-01f4-49b8-b70f-df2250618564 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:47.922985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5aa33-dd16-43b3-97b3-ac5a5d2a7ed8 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:47.922985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5aa33-dd16-43b3-97b3-ac5a5d2a7ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:47.923418 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5aa33-dd16-43b3-97b3-ac5a5d2a7ed8 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:47.925012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 229/442] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:47.930867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e54f5-69cd-4eac-90e3-49bd9c36a312 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:47.930867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e54f5-69cd-4eac-90e3-49bd9c36a312 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:47.930867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e54f5-69cd-4eac-90e3-49bd9c36a312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:47.958222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a61e54f5-69cd-4eac-90e3-49bd9c36a312 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:47.959423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a61e54f5-69cd-4eac-90e3-49bd9c36a312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:47.972457 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6c66d-01f4-49b8-b70f-df2250618564 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:48.000977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:48.015392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b356f-69ba-408c-8bb6-94b0aca66d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:48.016039 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b356f-69ba-408c-8bb6-94b0aca66d94 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:48.016322 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b356f-69ba-408c-8bb6-94b0aca66d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:48.016597 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b356f-69ba-408c-8bb6-94b0aca66d94 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:48.030000 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b356f-69ba-408c-8bb6-94b0aca66d94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:48.035501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-bd5b356f-69ba-408c-8bb6-94b0aca66d94 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:48.035501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:48.035501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:48.035501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:48.035501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:48.035501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:48.035501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:48.035501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:48.035501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:48.035501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:48.035501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:48.035501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:48.035501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:48.035501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:48.035501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:48.035501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:48.036528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:48.036528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:48.036528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:48.036528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:48.036528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:48.036528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:48.036528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:48.036528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:48.036528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:48.036528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:48.036528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:48.036528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:48.036528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:48.036528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:48.036528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:48.036528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:48.037535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:48.037535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:48.037535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:48.037535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:48.037535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:48.037535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:48.037535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:48.037535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:48.037535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:48.037535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:48.037535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:48.037535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:48.037535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 214/443] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:57:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:48.043906 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b356f-69ba-408c-8bb6-94b0aca66d94 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:48.049827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 230/444] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:47 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:57:48.074090 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e54f5-69cd-4eac-90e3-49bd9c36a312 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:48.131720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:48.168954 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b356f-69ba-408c-8bb6-94b0aca66d94 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:48.194465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e54f5-69cd-4eac-90e3-49bd9c36a312 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:48.203100 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2304d-2b66-4cb7-bf13-f521b33b46b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:48.203714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2304d-2b66-4cb7-bf13-f521b33b46b5 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:48.204000 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2304d-2b66-4cb7-bf13-f521b33b46b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:48.204280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2304d-2b66-4cb7-bf13-f521b33b46b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:48.205023 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2304d-2b66-4cb7-bf13-f521b33b46b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:48.205361 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2304d-2b66-4cb7-bf13-f521b33b46b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:48.206754 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2304d-2b66-4cb7-bf13-f521b33b46b5 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:48.215170 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 215/445] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:57:48 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:48.220645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2304d-2b66-4cb7-bf13-f521b33b46b5 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:48.229640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a07dc5-3683-495b-ab5b-19490ef940b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:48.230287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a07dc5-3683-495b-ab5b-19490ef940b6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:48.230572 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a07dc5-3683-495b-ab5b-19490ef940b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:48.230855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a07dc5-3683-495b-ab5b-19490ef940b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:48.240931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a07dc5-3683-495b-ab5b-19490ef940b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:48.247077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-e9a07dc5-3683-495b-ab5b-19490ef940b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:57:48.247077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:48.247077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:48.247077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:48.247077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:48.247077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:48.247077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:48.247077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:48.247077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:48.247077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:48.247077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:48.247077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:48.247077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:48.247077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:48.247077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:48.247077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:48.248112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:48.248112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:48.248112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:48.248112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:48.248112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:48.248112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:48.248112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:48.248112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:48.248112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:48.248112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:48.248112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:48.248112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:48.248112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:48.248112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:48.248112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:48.248112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:48.254027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:48.254027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:48.254027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:48.254027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:48.254027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:48.254027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:48.254027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:48.254027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:48.254027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:48.254027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:48.254027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:57:48.254027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:48.254027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 231/446] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:57:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:57:48.254698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a07dc5-3683-495b-ab5b-19490ef940b6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:48.267145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:48.319245 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2304d-2b66-4cb7-bf13-f521b33b46b5 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:48.352287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6398cb-cf44-4ac5-b114-4cde3c998dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:48.352832 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6398cb-cf44-4ac5-b114-4cde3c998dba None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:48.352949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a07dc5-3683-495b-ab5b-19490ef940b6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:48.353136 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6398cb-cf44-4ac5-b114-4cde3c998dba None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:48.353437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6398cb-cf44-4ac5-b114-4cde3c998dba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:48.354623 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:48.388155 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf303d1-3160-4eaf-9ff8-e63cce186f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:48.388765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf303d1-3160-4eaf-9ff8-e63cce186f34 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:48.389043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf303d1-3160-4eaf-9ff8-e63cce186f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:48.389342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf303d1-3160-4eaf-9ff8-e63cce186f34 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:48.390049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf303d1-3160-4eaf-9ff8-e63cce186f34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:48.390352 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf303d1-3160-4eaf-9ff8-e63cce186f34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:48.391696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf303d1-3160-4eaf-9ff8-e63cce186f34 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:48.399493 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 232/447] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:48 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:48.405038 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf303d1-3160-4eaf-9ff8-e63cce186f34 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:48.420316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6398cb-cf44-4ac5-b114-4cde3c998dba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:48.421805 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6398cb-cf44-4ac5-b114-4cde3c998dba None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:48.466463 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:48.505197 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf303d1-3160-4eaf-9ff8-e63cce186f34 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:48.521495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 216/448] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:48 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 08:57:48.526795 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6398cb-cf44-4ac5-b114-4cde3c998dba None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:48.539234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e45d44-1cb9-4729-bc1e-92784fd1a3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:48.539880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e45d44-1cb9-4729-bc1e-92784fd1a3ea None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:48.540160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e45d44-1cb9-4729-bc1e-92784fd1a3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:48.540434 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e45d44-1cb9-4729-bc1e-92784fd1a3ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:48.570976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:48.608198 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e45d44-1cb9-4729-bc1e-92784fd1a3ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:48.609595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e45d44-1cb9-4729-bc1e-92784fd1a3ea None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:48.626868 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6398cb-cf44-4ac5-b114-4cde3c998dba None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:48.661939 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6dc5a-fb32-4a2a-8953-320184319830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:48.662577 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6dc5a-fb32-4a2a-8953-320184319830 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:48.662858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6dc5a-fb32-4a2a-8953-320184319830 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:48.663141 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6dc5a-fb32-4a2a-8953-320184319830 None admin] PATH_INFO: `/v3/projects/51e776c25b1649278e436bb36fe083f4` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:48.664284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6dc5a-fb32-4a2a-8953-320184319830 None admin] RBAC: Authorizing `identity:get_project(project_id=51e776c25b1649278e436bb36fe083f4)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:48.666265 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6dc5a-fb32-4a2a-8953-320184319830 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:48.667921 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 217/449] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:48 2022] GET /identity/v3/projects/51e776c25b1649278e436bb36fe083f4 => generated 303 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:48.673252 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6dc5a-fb32-4a2a-8953-320184319830 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:48.688903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:48.709401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 233/450] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:48 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 08:57:48.715714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e45d44-1cb9-4729-bc1e-92784fd1a3ea None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:48.773547 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6dc5a-fb32-4a2a-8953-320184319830 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:48.777165 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:48.807341 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc310d-c4d9-483a-866d-88b74bd5d9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:48.808014 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc310d-c4d9-483a-866d-88b74bd5d9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:48.808298 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc310d-c4d9-483a-866d-88b74bd5d9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:48.808580 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc310d-c4d9-483a-866d-88b74bd5d9b8 None admin] PATH_INFO: `/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:48.809744 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc310d-c4d9-483a-866d-88b74bd5d9b8 None admin] RBAC: Authorizing `identity:get_group(group_id=f3b15961a0154a2abd8e3ba87e5b37b2)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:48.811184 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc310d-c4d9-483a-866d-88b74bd5d9b8 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:48.813084 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 218/451] 10.222.0.100 () {56 vars in 1185 bytes} [Tue Aug 2 08:57:48 2022] GET /identity/v3/groups/f3b15961a0154a2abd8e3ba87e5b37b2 => generated 227 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:48.816186 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e45d44-1cb9-4729-bc1e-92784fd1a3ea None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:48.819246 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc310d-c4d9-483a-866d-88b74bd5d9b8 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:48.849488 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96397c52-351a-41a8-a787-68a6ba56b2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:48.850133 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96397c52-351a-41a8-a787-68a6ba56b2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:48.850420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96397c52-351a-41a8-a787-68a6ba56b2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:48.850692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96397c52-351a-41a8-a787-68a6ba56b2b9 None admin] PATH_INFO: `/v3/projects/1b02845ab43a49ada6db1e8f52cf0026` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:48.851851 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96397c52-351a-41a8-a787-68a6ba56b2b9 None admin] RBAC: Authorizing `identity:get_project(project_id=1b02845ab43a49ada6db1e8f52cf0026)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:48.854142 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96397c52-351a-41a8-a787-68a6ba56b2b9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:48.855553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 234/452] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:57:48 2022] GET /identity/v3/projects/1b02845ab43a49ada6db1e8f52cf0026 => generated 345 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:48.860945 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96397c52-351a-41a8-a787-68a6ba56b2b9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:48.904242 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:48.921964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc310d-c4d9-483a-866d-88b74bd5d9b8 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:48.956796 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900d146-26e8-4b6d-b724-ad2b55489cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:48.957442 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900d146-26e8-4b6d-b724-ad2b55489cbd None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:48.957729 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900d146-26e8-4b6d-b724-ad2b55489cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:48.958002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900d146-26e8-4b6d-b724-ad2b55489cbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:48.958809 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8900d146-26e8-4b6d-b724-ad2b55489cbd None admin] RBAC: Adding query filter params (group.id=f3b15961a0154a2abd8e3ba87e5b37b2, role.id=bb986e49c38c40bab629ff32371b9358, scope.project.id=51e776c25b1649278e436bb36fe083f4) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:48.959120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8900d146-26e8-4b6d-b724-ad2b55489cbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:48.960542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8900d146-26e8-4b6d-b724-ad2b55489cbd None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:48.965825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96397c52-351a-41a8-a787-68a6ba56b2b9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:48.966446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 219/453] 10.222.0.100 () {56 vars in 1381 bytes} [Tue Aug 2 08:57:48 2022] GET /identity/v3/role_assignments?group.id=f3b15961a0154a2abd8e3ba87e5b37b2&scope.project.id=51e776c25b1649278e436bb36fe083f4&role.id=bb986e49c38c40bab629ff32371b9358 => generated 613 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:48.998972 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:49.004876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a5f81-2c42-41b5-a2a6-5aed0ed7381c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:49.005580 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a5f81-2c42-41b5-a2a6-5aed0ed7381c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:49.005885 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a5f81-2c42-41b5-a2a6-5aed0ed7381c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:49.006146 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a5f81-2c42-41b5-a2a6-5aed0ed7381c None admin] PATH_INFO: `/v3/groups/90ece6014584498485c079a870f3ffdf` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:49.007326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a5f81-2c42-41b5-a2a6-5aed0ed7381c None admin] RBAC: Authorizing `identity:get_group(group_id=90ece6014584498485c079a870f3ffdf)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:49.008789 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a5f81-2c42-41b5-a2a6-5aed0ed7381c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:49.010546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 235/454] 10.222.0.100 () {56 vars in 1185 bytes} [Tue Aug 2 08:57:48 2022] GET /identity/v3/groups/90ece6014584498485c079a870f3ffdf => generated 230 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:57:49.016657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900d146-26e8-4b6d-b724-ad2b55489cbd None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:49.121830 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900d146-26e8-4b6d-b724-ad2b55489cbd None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:49.156398 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc47482-e8be-46df-a20d-0c8841c12760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:49.157007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc47482-e8be-46df-a20d-0c8841c12760 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:49.157318 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc47482-e8be-46df-a20d-0c8841c12760 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:49.157597 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc47482-e8be-46df-a20d-0c8841c12760 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:49.158383 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc47482-e8be-46df-a20d-0c8841c12760 None admin] RBAC: Adding query filter params (group.id=90ece6014584498485c079a870f3ffdf, role.id=7cbff51867a7475d837afd51f6c12f11, scope.project.id=1b02845ab43a49ada6db1e8f52cf0026) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:49.158695 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc47482-e8be-46df-a20d-0c8841c12760 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:49.160153 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc47482-e8be-46df-a20d-0c8841c12760 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:49.166621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 220/455] 10.222.0.100 () {56 vars in 1381 bytes} [Tue Aug 2 08:57:49 2022] GET /identity/v3/role_assignments?group.id=90ece6014584498485c079a870f3ffdf&scope.project.id=1b02845ab43a49ada6db1e8f52cf0026&role.id=7cbff51867a7475d837afd51f6c12f11 => generated 613 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:51.529815 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345160bd-76b3-4b3a-8c50-86ed22ec9467 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:51.530332 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345160bd-76b3-4b3a-8c50-86ed22ec9467 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:51.530437 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345160bd-76b3-4b3a-8c50-86ed22ec9467 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:51.532010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 236/456] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:51.537197 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe68af4-c3dd-4788-896d-e51d77ac0732 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:51.538062 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe68af4-c3dd-4788-896d-e51d77ac0732 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:51.538637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe68af4-c3dd-4788-896d-e51d77ac0732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:51.564095 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-dbe68af4-c3dd-4788-896d-e51d77ac0732 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:51.565644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-dbe68af4-c3dd-4788-896d-e51d77ac0732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:51.568828 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:51.657883 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 221/457] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:51.698734 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345160bd-76b3-4b3a-8c50-86ed22ec9467 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:51.832586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345160bd-76b3-4b3a-8c50-86ed22ec9467 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:51.859619 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af9f89-a107-4361-bf5c-8f9f7a1bfe58 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:51.859925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af9f89-a107-4361-bf5c-8f9f7a1bfe58 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:51.860186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af9f89-a107-4361-bf5c-8f9f7a1bfe58 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:51.861820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 222/458] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:51.866775 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a79435-64a8-40b1-8d4e-19081bb07a75 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:51.867062 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a79435-64a8-40b1-8d4e-19081bb07a75 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:51.867351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a79435-64a8-40b1-8d4e-19081bb07a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:51.895566 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-42a79435-64a8-40b1-8d4e-19081bb07a75 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:51.896757 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-42a79435-64a8-40b1-8d4e-19081bb07a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:51.899455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561f191-7bb5-4777-ada7-85f1aed8b37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:51.900399 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561f191-7bb5-4777-ada7-85f1aed8b37e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:51.900689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561f191-7bb5-4777-ada7-85f1aed8b37e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:51.900966 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561f191-7bb5-4777-ada7-85f1aed8b37e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:51.929711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561f191-7bb5-4777-ada7-85f1aed8b37e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:51.932203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561f191-7bb5-4777-ada7-85f1aed8b37e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:51.959569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-6561f191-7bb5-4777-ada7-85f1aed8b37e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:51.959569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:51.959569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:51.959569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:51.959569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:51.959569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:51.959569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:51.959569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:51.959569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:51.959569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:51.959569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:51.959569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:51.959569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:57:51.959569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:57:51.959569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:57:51.960486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:57:51.960486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:51.960486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:51.960486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:51.960486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:51.960486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:51.960486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:51.960486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:51.960486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:51.960486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:51.960486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:51.960486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:51.960486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:51.960486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:51.960486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:51.960486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:51.960486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:51.961383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:57:51.961383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:57:51.961383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:51.961383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:51.967823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 237/459] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:51.968715 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561f191-7bb5-4777-ada7-85f1aed8b37e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:52.023739 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 223/460] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:57:52.025930 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c86408-3c78-4629-b6bd-6f6ed02ded1a None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:52.026423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c86408-3c78-4629-b6bd-6f6ed02ded1a None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:52.026894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c86408-3c78-4629-b6bd-6f6ed02ded1a None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:52.032718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 224/461] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:52 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:52.034261 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f23f9e-400d-4fa7-a852-e68d46486ee5 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:52.034780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f23f9e-400d-4fa7-a852-e68d46486ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:52.035254 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f23f9e-400d-4fa7-a852-e68d46486ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:52.059473 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-f4f23f9e-400d-4fa7-a852-e68d46486ee5 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:52.060775 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f4f23f9e-400d-4fa7-a852-e68d46486ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:52.066380 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:52.124185 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561f191-7bb5-4777-ada7-85f1aed8b37e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:52.136618 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:52.143007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 225/462] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:52 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:52.143917 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f23f9e-400d-4fa7-a852-e68d46486ee5 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:52.172922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572c127-0494-47c4-9215-82dce30f13b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:52.173938 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572c127-0494-47c4-9215-82dce30f13b5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:52.174426 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572c127-0494-47c4-9215-82dce30f13b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:52.174919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572c127-0494-47c4-9215-82dce30f13b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:52.175812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b572c127-0494-47c4-9215-82dce30f13b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:52.176327 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b572c127-0494-47c4-9215-82dce30f13b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:52.177925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b572c127-0494-47c4-9215-82dce30f13b5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:52.190870 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 238/463] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:57:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:52.192017 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572c127-0494-47c4-9215-82dce30f13b5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:52.255719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f23f9e-400d-4fa7-a852-e68d46486ee5 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:52.294412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff611a-496a-47d7-bc10-2bddf699e3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:52.295158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff611a-496a-47d7-bc10-2bddf699e3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:52.295445 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff611a-496a-47d7-bc10-2bddf699e3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:52.295723 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff611a-496a-47d7-bc10-2bddf699e3f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:52.307947 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff611a-496a-47d7-bc10-2bddf699e3f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:52.309530 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff611a-496a-47d7-bc10-2bddf699e3f1 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:52.328863 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572c127-0494-47c4-9215-82dce30f13b5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:52.329506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-cbff611a-496a-47d7-bc10-2bddf699e3f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:52.329506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:52.329506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:52.329506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:52.329506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:52.329506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:52.329506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:52.329506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:52.329506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:52.329506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:52.329506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:52.329506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:52.329506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:57:52.329506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:57:52.329506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:57:52.330321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:57:52.330321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:52.330321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:52.330321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:52.330321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:52.330321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:52.330321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:52.330321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:52.330321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:52.330321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:52.330321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:52.330321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:52.330321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:52.330321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:52.330321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:52.330321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:52.330321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:52.331256 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:57:52.331256 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:57:52.331256 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:52.331256 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:52.331256 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 226/464] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:52.331527 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff611a-496a-47d7-bc10-2bddf699e3f1 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:52.350548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:52.380878 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f3796-6bf4-45d8-b5a9-346cb0c4bd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:52.381894 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f3796-6bf4-45d8-b5a9-346cb0c4bd94 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:52.382377 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f3796-6bf4-45d8-b5a9-346cb0c4bd94 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:52.382867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f3796-6bf4-45d8-b5a9-346cb0c4bd94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:52.392181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:52.395062 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f3796-6bf4-45d8-b5a9-346cb0c4bd94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:52.397497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f3796-6bf4-45d8-b5a9-346cb0c4bd94 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:52.412476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-d87f3796-6bf4-45d8-b5a9-346cb0c4bd94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:52.412476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:52.412476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:52.412476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:52.412476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:52.412476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:52.412476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:52.412476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:52.412476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:52.412476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:52.412476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:52.412476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:52.412476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:57:52.412476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:57:52.412476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:57:52.413423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:57:52.413423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:52.413423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:52.413423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:52.413423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:52.413423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:52.413423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:52.413423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:52.413423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:52.413423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:52.413423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:52.413423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:52.413423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:52.413423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:52.413423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:52.413423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:52.413423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:52.414274 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:57:52.414274 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:57:52.414274 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:52.414274 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:52.414274 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 239/465] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:52.419796 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f3796-6bf4-45d8-b5a9-346cb0c4bd94 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:52.446762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff611a-496a-47d7-bc10-2bddf699e3f1 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:52.481929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a92128-33d4-408f-b377-5a93ab4fad17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:52.482543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a92128-33d4-408f-b377-5a93ab4fad17 None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:52.482824 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a92128-33d4-408f-b377-5a93ab4fad17 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:52.483105 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a92128-33d4-408f-b377-5a93ab4fad17 None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:52.483778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a92128-33d4-408f-b377-5a93ab4fad17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:52.485245 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a92128-33d4-408f-b377-5a93ab4fad17 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:52.510135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 227/466] 10.222.0.100 () {60 vars in 1137 bytes} [Tue Aug 2 08:57:52 2022] POST /identity/v3/users => generated 247 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:57:52.522673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6df138-e68e-41bc-b26c-236c04717236 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:52.523009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6df138-e68e-41bc-b26c-236c04717236 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:52.523235 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6df138-e68e-41bc-b26c-236c04717236 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:52.524739 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 228/467] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:52.525395 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6df138-e68e-41bc-b26c-236c04717236 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:52.542960 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f3796-6bf4-45d8-b5a9-346cb0c4bd94 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:52.599478 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fa6a2-2c66-46ed-ad72-d99bfb169e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:52.600625 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fa6a2-2c66-46ed-ad72-d99bfb169e91 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:52.600923 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fa6a2-2c66-46ed-ad72-d99bfb169e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:52.601570 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fa6a2-2c66-46ed-ad72-d99bfb169e91 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:52.602304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9fa6a2-2c66-46ed-ad72-d99bfb169e91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:52.603385 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9fa6a2-2c66-46ed-ad72-d99bfb169e91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:52.605648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9fa6a2-2c66-46ed-ad72-d99bfb169e91 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:52.623187 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 240/468] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:57:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:52.624511 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e9271-3dca-47ee-8742-a4958f595120 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:52.624888 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e9271-3dca-47ee-8742-a4958f595120 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:52.625272 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e9271-3dca-47ee-8742-a4958f595120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:52.645311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-be1e9271-3dca-47ee-8742-a4958f595120 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:52.646612 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-be1e9271-3dca-47ee-8742-a4958f595120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:52.667909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:52.669171 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6df138-e68e-41bc-b26c-236c04717236 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:52.681923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:52.714885 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66132a-194a-4b48-b6c1-fc8fad27be35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:52.715687 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66132a-194a-4b48-b6c1-fc8fad27be35 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:52.716144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66132a-194a-4b48-b6c1-fc8fad27be35 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:52.716512 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66132a-194a-4b48-b6c1-fc8fad27be35 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:52.717335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66132a-194a-4b48-b6c1-fc8fad27be35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:52.717735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66132a-194a-4b48-b6c1-fc8fad27be35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:52.719278 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66132a-194a-4b48-b6c1-fc8fad27be35 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:52.732500 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 229/469] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:57:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:52.734163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66132a-194a-4b48-b6c1-fc8fad27be35 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:52.740956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 241/470] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:52 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:52.741720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e9271-3dca-47ee-8742-a4958f595120 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:52.865051 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66132a-194a-4b48-b6c1-fc8fad27be35 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:52.870293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e9271-3dca-47ee-8742-a4958f595120 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:52.909950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd29cdb-668d-4d63-8ab6-e50ef6e25973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:52.910503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd29cdb-668d-4d63-8ab6-e50ef6e25973 None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:52.910789 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd29cdb-668d-4d63-8ab6-e50ef6e25973 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:52.911068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd29cdb-668d-4d63-8ab6-e50ef6e25973 None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:52.912047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd29cdb-668d-4d63-8ab6-e50ef6e25973 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:52.913526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd29cdb-668d-4d63-8ab6-e50ef6e25973 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:52.922407 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb348a-ab35-4f2a-8c85-49b62b2fa9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:52.923303 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb348a-ab35-4f2a-8c85-49b62b2fa9c3 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:52.923814 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb348a-ab35-4f2a-8c85-49b62b2fa9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:52.924290 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb348a-ab35-4f2a-8c85-49b62b2fa9c3 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:52.925204 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb348a-ab35-4f2a-8c85-49b62b2fa9c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:52.926902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb348a-ab35-4f2a-8c85-49b62b2fa9c3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:52.940895 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 230/471] 10.222.0.100 () {60 vars in 1137 bytes} [Tue Aug 2 08:57:52 2022] POST /identity/v3/users => generated 249 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:57:52.943114 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd29cdb-668d-4d63-8ab6-e50ef6e25973 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:52.954713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 242/472] 10.222.0.100 () {60 vars in 1137 bytes} [Tue Aug 2 08:57:52 2022] POST /identity/v3/users => generated 249 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:57:52.981203 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:53.077390 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd29cdb-668d-4d63-8ab6-e50ef6e25973 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:53.118370 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f9429-29f7-4656-84b1-e2bbef7ce019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:53.119647 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f9429-29f7-4656-84b1-e2bbef7ce019 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:53.119953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f9429-29f7-4656-84b1-e2bbef7ce019 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:53.120225 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f9429-29f7-4656-84b1-e2bbef7ce019 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:53.132430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f9429-29f7-4656-84b1-e2bbef7ce019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:53.135288 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f9429-29f7-4656-84b1-e2bbef7ce019 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:53.152814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-e61f9429-29f7-4656-84b1-e2bbef7ce019 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:53.152814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:53.152814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:53.152814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:53.152814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:53.152814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:53.152814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:53.152814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:53.152814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:53.152814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:53.152814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:53.152814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:53.152814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:57:53.152814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:57:53.152814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:57:53.153814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:57:53.153814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:53.153814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:53.153814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:53.153814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:53.153814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:53.153814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:53.153814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:53.153814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:53.153814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:53.153814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:53.153814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:53.153814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:53.153814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:53.153814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:53.153814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:53.153814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:53.154673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:57:53.154673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:57:53.154673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:53.154673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:53.154673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 231/473] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:53.162696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb348a-ab35-4f2a-8c85-49b62b2fa9c3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:53.172315 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e43e4d-cf27-489d-9b10-d466b7bc4b25 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:53.172617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e43e4d-cf27-489d-9b10-d466b7bc4b25 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:53.172853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e43e4d-cf27-489d-9b10-d466b7bc4b25 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:53.174718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 232/474] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:53.179787 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f4b95-3b47-4fbe-a1be-475845b9c01d None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:53.180084 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f4b95-3b47-4fbe-a1be-475845b9c01d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:53.180340 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f4b95-3b47-4fbe-a1be-475845b9c01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:53.185771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:53.207854 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-a36f4b95-3b47-4fbe-a1be-475845b9c01d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:53.209515 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-a36f4b95-3b47-4fbe-a1be-475845b9c01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:53.296808 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 233/475] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:53 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:57:53.306795 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb348a-ab35-4f2a-8c85-49b62b2fa9c3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:53.323296 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f4b95-3b47-4fbe-a1be-475845b9c01d None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:53.334099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:53.347266 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7cf9e-ab71-460a-b5a8-1c9797be2043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:53.347903 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7cf9e-ab71-460a-b5a8-1c9797be2043 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:53.348191 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7cf9e-ab71-460a-b5a8-1c9797be2043 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:53.348472 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7cf9e-ab71-460a-b5a8-1c9797be2043 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:53.349528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7cf9e-ab71-460a-b5a8-1c9797be2043 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:53.349637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7cf9e-ab71-460a-b5a8-1c9797be2043 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:53.351024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7cf9e-ab71-460a-b5a8-1c9797be2043 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:53.365446 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 243/476] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:57:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:53.371870 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7cf9e-ab71-460a-b5a8-1c9797be2043 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:53.436996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f4b95-3b47-4fbe-a1be-475845b9c01d None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:53.469188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:53.474481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ca05d-1289-4e27-a7a6-a48b049810ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:53.475399 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ca05d-1289-4e27-a7a6-a48b049810ba None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:53.475931 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ca05d-1289-4e27-a7a6-a48b049810ba None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:53.476470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ca05d-1289-4e27-a7a6-a48b049810ba None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:53.477451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ca05d-1289-4e27-a7a6-a48b049810ba None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:53.479975 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ca05d-1289-4e27-a7a6-a48b049810ba None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:53.488222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.sql.core [None req-4c9ca05d-1289-4e27-a7a6-a48b049810ba None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 02 08:57:53.488222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 02 08:57:53.488222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [parameters: {'id': '9ced5983805846f6a8a7cbde4b2096d0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 02 08:57:53.488222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80361) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 02 08:57:53.506409 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7cf9e-ab71-460a-b5a8-1c9797be2043 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:53.507492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-4c9ca05d-1289-4e27-a7a6-a48b049810ba None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 02 08:57:53.507492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:53.507492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 08:57:53.507492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 08:57:53.507492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 08:57:53.507492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 08:57:53.507492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 08:57:53.507492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 08:57:53.507492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 08:57:53.507492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application conn.query(q) Aug 02 08:57:53.507492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 08:57:53.507492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 08:57:53.507492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 08:57:53.507492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application result.read() Aug 02 08:57:53.507492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 08:57:53.508437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 08:57:53.508437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 08:57:53.508437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 08:57:53.508437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 08:57:53.508437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 08:57:53.508437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 08:57:53.508437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 08:57:53.508437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 02 08:57:53.508437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:53.508437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 02 08:57:53.508437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:53.508437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:53.508437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 02 08:57:53.508437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 02 08:57:53.508437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 02 08:57:53.508437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return ref.to_dict() Aug 02 08:57:53.508437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 08:57:53.508437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application next(self.gen) Aug 02 08:57:53.509491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 02 08:57:53.509491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application yield resource Aug 02 08:57:53.509491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 08:57:53.509491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application next(self.gen) Aug 02 08:57:53.509491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 02 08:57:53.509491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self.session.rollback() Aug 02 08:57:53.509491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 02 08:57:53.509491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self.force_reraise() Aug 02 08:57:53.509491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 02 08:57:53.509491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise self.value Aug 02 08:57:53.509491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 02 08:57:53.509491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 02 08:57:53.509491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 02 08:57:53.509491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application session.commit() Aug 02 08:57:53.509491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 02 08:57:53.509491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 02 08:57:53.509491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 02 08:57:53.510365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._prepare_impl() Aug 02 08:57:53.510365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 02 08:57:53.510365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self.session.flush() Aug 02 08:57:53.510365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 02 08:57:53.510365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._flush(objects) Aug 02 08:57:53.510365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 02 08:57:53.510365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 02 08:57:53.510365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 02 08:57:53.510365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application compat.raise_( Aug 02 08:57:53.510365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 08:57:53.510365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception Aug 02 08:57:53.510365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 02 08:57:53.510365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application flush_context.execute() Aug 02 08:57:53.510365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 02 08:57:53.510365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rec.execute(self) Aug 02 08:57:53.510365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 02 08:57:53.510365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 02 08:57:53.511243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 02 08:57:53.511243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 02 08:57:53.511243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 02 08:57:53.511243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 02 08:57:53.511243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 02 08:57:53.511243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 02 08:57:53.511243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 02 08:57:53.511243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 02 08:57:53.511243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 02 08:57:53.511243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 02 08:57:53.511243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 02 08:57:53.511243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 02 08:57:53.511243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 02 08:57:53.511243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 02 08:57:53.511243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 08:57:53.511243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception Aug 02 08:57:53.516648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 08:57:53.516648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 08:57:53.516648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 08:57:53.516648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 08:57:53.516648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 08:57:53.516648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 08:57:53.516648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 08:57:53.516648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application conn.query(q) Aug 02 08:57:53.516648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 08:57:53.516648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 08:57:53.516648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 08:57:53.516648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application result.read() Aug 02 08:57:53.516648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 08:57:53.516648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 08:57:53.516648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 08:57:53.516648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 08:57:53.518621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 08:57:53.518621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 08:57:53.518621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 08:57:53.518621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 08:57:53.518621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 02 08:57:53.518621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 02 08:57:53.518621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [parameters: {'id': '9ced5983805846f6a8a7cbde4b2096d0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 02 08:57:53.518621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 02 08:57:53.518621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:53.518621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:57:53.518621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:53.518621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:53.518621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:53.518621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:53.518621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:53.518621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:53.520583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:53.520583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:53.520583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:53.520583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:53.520583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:53.520583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:53.520583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 02 08:57:53.520583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 02 08:57:53.520583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:53.520583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:53.520583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 02 08:57:53.520583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 02 08:57:53.520583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 02 08:57:53.520583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 02 08:57:53.520583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 02 08:57:53.520583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:53.522646 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 234/477] 10.222.0.100 () {60 vars in 1136 bytes} [Tue Aug 2 08:57:53 2022] POST /identity/v3/roles => generated 144 bytes in 188 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:57:53.522646 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ca05d-1289-4e27-a7a6-a48b049810ba None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:53.582193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2d061-3693-4958-8cfb-b64f0241e87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:53.582860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2d061-3693-4958-8cfb-b64f0241e87b None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:53.583093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2d061-3693-4958-8cfb-b64f0241e87b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:53.583831 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2d061-3693-4958-8cfb-b64f0241e87b None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:53.584304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2d061-3693-4958-8cfb-b64f0241e87b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:53.586483 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2d061-3693-4958-8cfb-b64f0241e87b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:53.629989 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 244/478] 10.222.0.100 () {60 vars in 1137 bytes} [Tue Aug 2 08:57:53 2022] POST /identity/v3/users => generated 250 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:57:53.654983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:53.714587 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ca05d-1289-4e27-a7a6-a48b049810ba None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:53.771173 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2baef-74e5-4dc0-b82d-2d1880cc5c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:53.771837 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2baef-74e5-4dc0-b82d-2d1880cc5c3a None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:53.772121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2baef-74e5-4dc0-b82d-2d1880cc5c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:53.772397 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2baef-74e5-4dc0-b82d-2d1880cc5c3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:53.790130 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2baef-74e5-4dc0-b82d-2d1880cc5c3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:53.795863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-16f2baef-74e5-4dc0-b82d-2d1880cc5c3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:53.795863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:53.795863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:53.795863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:53.795863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:53.795863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:53.795863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:53.795863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:53.795863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:53.795863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:53.795863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:53.795863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:53.795863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:53.795863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:53.795863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:53.795863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:53.796776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:53.796776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:53.796776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:53.796776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:53.796776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:53.796776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:53.796776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:53.796776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:53.796776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:53.796776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:53.796776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:53.796776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:53.796776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:53.796776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:53.796776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:53.796776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:53.797632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:53.797632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:53.797632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:53.797632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:53.797632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:53.797632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:53.797632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:53.797632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:53.797632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:53.797632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:53.797632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:57:53.797632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:53.797632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 235/479] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:57:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:57:53.803727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2d061-3693-4958-8cfb-b64f0241e87b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:53.915184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2d061-3693-4958-8cfb-b64f0241e87b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:53.945208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:53.955147 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea1791-8c81-48e2-8a06-f87eb771d291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:53.955772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea1791-8c81-48e2-8a06-f87eb771d291 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:53.956055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea1791-8c81-48e2-8a06-f87eb771d291 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:53.956331 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea1791-8c81-48e2-8a06-f87eb771d291 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:53.957067 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea1791-8c81-48e2-8a06-f87eb771d291 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:53.957394 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea1791-8c81-48e2-8a06-f87eb771d291 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:53.958753 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea1791-8c81-48e2-8a06-f87eb771d291 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:53.971525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 245/480] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:57:53 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:54.907553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451baa7-2856-400e-90cd-cb4616704372 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:54.908546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451baa7-2856-400e-90cd-cb4616704372 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:54.909030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451baa7-2856-400e-90cd-cb4616704372 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:54.910937 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 236/481] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:54 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:54.916994 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f169b-4cd4-4b36-8601-60b4fd69da5d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:54.917900 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f169b-4cd4-4b36-8601-60b4fd69da5d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:54.918391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f169b-4cd4-4b36-8601-60b4fd69da5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:54.937737 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-769f169b-4cd4-4b36-8601-60b4fd69da5d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:54.939249 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-769f169b-4cd4-4b36-8601-60b4fd69da5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:55.009167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 246/482] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:55.032486 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451baa7-2856-400e-90cd-cb4616704372 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:55.143178 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451baa7-2856-400e-90cd-cb4616704372 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:55.174849 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:55.183678 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055e2a9-aad2-4e57-8fb5-cf030fae6abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:55.184738 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055e2a9-aad2-4e57-8fb5-cf030fae6abe None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:55.185247 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055e2a9-aad2-4e57-8fb5-cf030fae6abe None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:55.185735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055e2a9-aad2-4e57-8fb5-cf030fae6abe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:55.197630 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a055e2a9-aad2-4e57-8fb5-cf030fae6abe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:55.202858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-a055e2a9-aad2-4e57-8fb5-cf030fae6abe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:57:55.202858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:55.202858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:55.202858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:55.202858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:55.202858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:55.202858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:55.202858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:55.202858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:55.202858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:55.202858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:55.202858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:55.202858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:55.202858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:55.202858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:55.202858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:55.203816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:55.203816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:55.203816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:55.203816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:55.203816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:55.203816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:55.203816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:55.203816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:55.203816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:55.203816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:55.203816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:55.203816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:55.203816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:55.203816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:55.203816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:55.203816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:55.204601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:55.204601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:55.204601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:55.204601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:55.204601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:55.204601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:55.204601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:55.204601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:55.204601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:55.204601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:55.204601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:57:55.204601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:55.207252 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 237/483] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:57:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:55.213837 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f169b-4cd4-4b36-8601-60b4fd69da5d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:55.336408 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f169b-4cd4-4b36-8601-60b4fd69da5d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:55.359783 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:55.386376 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6ed56-3501-4c73-9bb3-99441ac67d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:55.387016 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6ed56-3501-4c73-9bb3-99441ac67d29 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:55.387303 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6ed56-3501-4c73-9bb3-99441ac67d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:55.387585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6ed56-3501-4c73-9bb3-99441ac67d29 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:55.388311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6ed56-3501-4c73-9bb3-99441ac67d29 None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:55.388621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6ed56-3501-4c73-9bb3-99441ac67d29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:55.390013 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6ed56-3501-4c73-9bb3-99441ac67d29 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:55.398805 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 247/484] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 08:57:55 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:55.404587 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055e2a9-aad2-4e57-8fb5-cf030fae6abe None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:55.577816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055e2a9-aad2-4e57-8fb5-cf030fae6abe None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:55.631001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ba23d-cf78-4791-8032-417bc47485d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:55.631671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ba23d-cf78-4791-8032-417bc47485d2 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:55.631955 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ba23d-cf78-4791-8032-417bc47485d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:55.632227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ba23d-cf78-4791-8032-417bc47485d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:55.638716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a62edd-09a2-4fc1-b3b2-bec2cd011dc5 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:55.639021 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a62edd-09a2-4fc1-b3b2-bec2cd011dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:55.639292 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a62edd-09a2-4fc1-b3b2-bec2cd011dc5 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:55.640866 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 248/485] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:55.647115 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52335c-34e6-441f-95aa-429febf51fd4 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:55.647399 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52335c-34e6-441f-95aa-429febf51fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:55.647785 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52335c-34e6-441f-95aa-429febf51fd4 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:55.651202 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ba23d-cf78-4791-8032-417bc47485d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:55.651328 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 249/486] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:55.651328 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c88b0-f8ec-418a-b69f-cc97125b384d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:55.651460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ba23d-cf78-4791-8032-417bc47485d2 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:55.651545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c88b0-f8ec-418a-b69f-cc97125b384d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:55.651545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c88b0-f8ec-418a-b69f-cc97125b384d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:55.667780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-6d6ba23d-cf78-4791-8032-417bc47485d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:55.667780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:55.667780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:55.667780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:55.667780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:55.667780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:55.667780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:55.667780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:55.667780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:55.667780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:55.667780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:55.667780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:55.667780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:57:55.667780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:57:55.667780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:57:55.668688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:57:55.668688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:55.668688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:55.668688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:55.668688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:55.668688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:55.668688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:55.668688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:55.668688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:55.668688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:55.668688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:55.668688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:55.668688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:55.668688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:55.668688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:55.668688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:55.668688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:55.669590 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:57:55.669590 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:57:55.669590 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:55.669590 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:55.671732 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 238/487] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:55.673103 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863390a9-5cfe-4cbe-ae3a-eb4b52be094c None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:55.673638 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863390a9-5cfe-4cbe-ae3a-eb4b52be094c None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:55.673742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-613c88b0-f8ec-418a-b69f-cc97125b384d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:55.674202 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863390a9-5cfe-4cbe-ae3a-eb4b52be094c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:55.674871 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-613c88b0-f8ec-418a-b69f-cc97125b384d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:55.693786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-863390a9-5cfe-4cbe-ae3a-eb4b52be094c None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:55.695388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-863390a9-5cfe-4cbe-ae3a-eb4b52be094c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:55.734065 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:55.748840 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 250/488] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:55 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:55.750017 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c88b0-f8ec-418a-b69f-cc97125b384d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:55.787398 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 239/489] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:55 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:55.793848 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863390a9-5cfe-4cbe-ae3a-eb4b52be094c None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:55.881327 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c88b0-f8ec-418a-b69f-cc97125b384d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:55.894025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:55.904552 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863390a9-5cfe-4cbe-ae3a-eb4b52be094c None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:55.922975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880726f-efd1-4c52-939f-6007d9e203de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:55.923722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880726f-efd1-4c52-939f-6007d9e203de None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:55.924111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880726f-efd1-4c52-939f-6007d9e203de None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:55.924489 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880726f-efd1-4c52-939f-6007d9e203de None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:55.925341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880726f-efd1-4c52-939f-6007d9e203de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:55.925756 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880726f-efd1-4c52-939f-6007d9e203de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:55.927313 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880726f-efd1-4c52-939f-6007d9e203de None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:55.941224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 251/490] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:57:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:55.942398 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880726f-efd1-4c52-939f-6007d9e203de None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:55.950799 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d54dcc-d9ff-424f-998a-acc2b64bdf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:55.952016 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d54dcc-d9ff-424f-998a-acc2b64bdf21 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:55.952454 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d54dcc-d9ff-424f-998a-acc2b64bdf21 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:55.952880 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d54dcc-d9ff-424f-998a-acc2b64bdf21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:55.964949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d54dcc-d9ff-424f-998a-acc2b64bdf21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:55.970363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-42d54dcc-d9ff-424f-998a-acc2b64bdf21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:57:55.970363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:55.970363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:55.970363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:55.970363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:55.970363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:55.970363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:55.970363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:55.970363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:55.970363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:55.970363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:55.970363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:55.970363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:55.970363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:55.970363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:55.970363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:55.971286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:55.971286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:55.971286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:55.971286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:55.971286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:55.971286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:55.971286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:55.971286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:55.971286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:55.971286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:55.971286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:55.971286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:55.971286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:55.971286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:55.971286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:55.971286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:55.972108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:55.972108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:55.972108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:55.972108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:55.972108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:55.972108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:55.972108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:55.972108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:55.972108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:55.972108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:55.972108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:57:55.972108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:55.974959 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 240/491] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:57:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:57:55.980708 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d54dcc-d9ff-424f-998a-acc2b64bdf21 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:56.015802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:56.054934 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880726f-efd1-4c52-939f-6007d9e203de None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:56.094412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d54dcc-d9ff-424f-998a-acc2b64bdf21 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:56.094971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414db1e8-8f06-42f1-a22c-2a9244e7753f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:56.095698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414db1e8-8f06-42f1-a22c-2a9244e7753f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:56.096062 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414db1e8-8f06-42f1-a22c-2a9244e7753f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:56.096416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414db1e8-8f06-42f1-a22c-2a9244e7753f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:56.108376 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414db1e8-8f06-42f1-a22c-2a9244e7753f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:56.114351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-414db1e8-8f06-42f1-a22c-2a9244e7753f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:57:56.114351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:56.114351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:56.114351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:56.114351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:56.114351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:56.114351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:56.114351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:56.114351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:56.114351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:56.114351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:56.114351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:56.114351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:56.114351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:56.114351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:56.114351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:56.115232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:56.115232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:56.115232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:56.115232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:56.115232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:56.115232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:56.115232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:56.115232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:56.115232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:56.115232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:56.115232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:56.115232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:56.115232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:56.115232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:56.115232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:56.115232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:56.116041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:56.116041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:56.116041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:56.116041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:56.116041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:56.116041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:56.116041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:56.116041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:56.116041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:56.116041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:56.116041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:57:56.116041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:56.121058 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 252/492] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:57:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:56.122256 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414db1e8-8f06-42f1-a22c-2a9244e7753f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:56.138576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad329d03-c700-4e52-9d6e-ccc61b49cef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:56.139553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad329d03-c700-4e52-9d6e-ccc61b49cef2 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:56.140037 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad329d03-c700-4e52-9d6e-ccc61b49cef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:56.140577 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad329d03-c700-4e52-9d6e-ccc61b49cef2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:56.149097 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad329d03-c700-4e52-9d6e-ccc61b49cef2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:56.151024 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad329d03-c700-4e52-9d6e-ccc61b49cef2 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:56.157579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:56.169393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-ad329d03-c700-4e52-9d6e-ccc61b49cef2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:57:56.169393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:56.169393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:56.169393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:56.169393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:56.169393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:56.169393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:56.169393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:56.169393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:56.169393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:56.169393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:56.169393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:56.169393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:57:56.169393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:57:56.169393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:57:56.169393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:57:56.170461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:56.170461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:56.170461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:57:56.170461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:57:56.170461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:57:56.170461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:57:56.170461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:56.170461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:56.170461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:56.170461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:56.170461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:56.170461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:56.170461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:56.170461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:56.170461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:56.170461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:56.170461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:56.170461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:56.171367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:56.171367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:56.171367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:57:56.171367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:57:56.171367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:57:56.171367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:57:56.171367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:57:56.171367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:57:56.171367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:57:56.171367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:56.174246 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 241/493] 10.222.0.100 () {56 vars in 1098 bytes} [Tue Aug 2 08:57:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:56.175403 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad329d03-c700-4e52-9d6e-ccc61b49cef2 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:56.236173 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414db1e8-8f06-42f1-a22c-2a9244e7753f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:56.274357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1fb36-39bb-4c98-a831-3f23419c0bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:56.275164 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1fb36-39bb-4c98-a831-3f23419c0bcb None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:56.275455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1fb36-39bb-4c98-a831-3f23419c0bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:56.275733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1fb36-39bb-4c98-a831-3f23419c0bcb None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:56.276648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1fb36-39bb-4c98-a831-3f23419c0bcb None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:56.276876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1fb36-39bb-4c98-a831-3f23419c0bcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:56.278347 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1fb36-39bb-4c98-a831-3f23419c0bcb None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:56.281064 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:56.293196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 253/494] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 08:57:56 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:56.293196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1fb36-39bb-4c98-a831-3f23419c0bcb None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:56.295866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad329d03-c700-4e52-9d6e-ccc61b49cef2 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:56.334021 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed2eae-568e-491c-889e-a9cbb96d962f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:56.334916 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed2eae-568e-491c-889e-a9cbb96d962f None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:56.335357 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed2eae-568e-491c-889e-a9cbb96d962f None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:56.335778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed2eae-568e-491c-889e-a9cbb96d962f None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:56.336794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ed2eae-568e-491c-889e-a9cbb96d962f None admin] RBAC: Adding query filter params (name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:56.337949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ed2eae-568e-491c-889e-a9cbb96d962f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:56.339648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ed2eae-568e-491c-889e-a9cbb96d962f None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:56.349298 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 242/495] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 08:57:56 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:56.350410 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed2eae-568e-491c-889e-a9cbb96d962f None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:56.399113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:56.409002 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1fb36-39bb-4c98-a831-3f23419c0bcb None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:56.445239 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db5d63-006c-4f8e-926f-317965627a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:56.445862 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db5d63-006c-4f8e-926f-317965627a35 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:56.446151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db5d63-006c-4f8e-926f-317965627a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:56.447043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db5d63-006c-4f8e-926f-317965627a35 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:56.447984 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db5d63-006c-4f8e-926f-317965627a35 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:56.448298 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db5d63-006c-4f8e-926f-317965627a35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:56.449811 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db5d63-006c-4f8e-926f-317965627a35 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:56.461294 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed2eae-568e-491c-889e-a9cbb96d962f None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:56.489281 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 254/496] 10.222.0.100 () {56 vars in 1138 bytes} [Tue Aug 2 08:57:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:56.489958 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db5d63-006c-4f8e-926f-317965627a35 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:56.500445 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e586e-0c66-4465-912e-8e63ce480dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:56.501395 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e586e-0c66-4465-912e-8e63ce480dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:56.501686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e586e-0c66-4465-912e-8e63ce480dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:56.501967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e586e-0c66-4465-912e-8e63ce480dc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:56.513256 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e586e-0c66-4465-912e-8e63ce480dc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:56.514719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e586e-0c66-4465-912e-8e63ce480dc2 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:56.529760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-b14e586e-0c66-4465-912e-8e63ce480dc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:56.529760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:56.529760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:56.529760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:56.529760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:56.529760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:56.529760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:56.529760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:56.529760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:56.529760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:56.529760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:56.529760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:56.529760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:57:56.529760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:57:56.529760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:57:56.530644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:57:56.530644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:56.530644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:56.530644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:56.530644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:56.530644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:56.530644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:56.530644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:56.530644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:56.530644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:56.530644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:56.530644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:56.530644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:56.530644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:56.530644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:56.530644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:56.530644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:56.531525 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:57:56.531525 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:57:56.531525 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:56.531525 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:56.531525 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 243/497] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:56.532209 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97f257-8ea9-4152-ab46-06bb7a2a2892 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:56.532463 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97f257-8ea9-4152-ab46-06bb7a2a2892 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:56.532742 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97f257-8ea9-4152-ab46-06bb7a2a2892 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:56.537144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 244/498] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:56 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:56.537144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97f257-8ea9-4152-ab46-06bb7a2a2892 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:56.547365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:56.596597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db5d63-006c-4f8e-926f-317965627a35 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:56.630440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f0771-065d-492d-8de9-48f24dc35002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:56.631079 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f0771-065d-492d-8de9-48f24dc35002 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:56.631369 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f0771-065d-492d-8de9-48f24dc35002 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:56.631644 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f0771-065d-492d-8de9-48f24dc35002 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:56.641288 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97f257-8ea9-4152-ab46-06bb7a2a2892 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:56.642186 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f0771-065d-492d-8de9-48f24dc35002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:56.643646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f0771-065d-492d-8de9-48f24dc35002 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:56.658533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-443f0771-065d-492d-8de9-48f24dc35002 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:56.658533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:56.658533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:56.658533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:56.658533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:56.658533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:56.658533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:56.658533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:56.658533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:56.658533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:56.658533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:56.658533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:56.658533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:57:56.658533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:57:56.658533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:57:56.659384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:57:56.659384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:56.659384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:56.659384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:56.659384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:56.659384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:56.659384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:56.659384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:56.659384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:56.659384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:56.659384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:56.659384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:56.659384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:56.659384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:56.659384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:56.659384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:56.659384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:56.660233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:57:56.660233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:57:56.660233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:56.660233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:56.660233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 255/499] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:56.660521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f0771-065d-492d-8de9-48f24dc35002 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:56.666627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:56.675318 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32db4f-a037-4688-95fa-318aae8faeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:56.675948 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32db4f-a037-4688-95fa-318aae8faeef None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:56.676234 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32db4f-a037-4688-95fa-318aae8faeef None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:56.676511 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32db4f-a037-4688-95fa-318aae8faeef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:56.742311 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32db4f-a037-4688-95fa-318aae8faeef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:56.744496 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32db4f-a037-4688-95fa-318aae8faeef None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:56.752420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:56.764383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f0771-065d-492d-8de9-48f24dc35002 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:56.798070 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c3955-f1f6-4f1f-9239-f95476e2936a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:56.798692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c3955-f1f6-4f1f-9239-f95476e2936a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:56.798982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c3955-f1f6-4f1f-9239-f95476e2936a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:56.799260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c3955-f1f6-4f1f-9239-f95476e2936a None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:56.799949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c3955-f1f6-4f1f-9239-f95476e2936a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:56.800260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c3955-f1f6-4f1f-9239-f95476e2936a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:56.801685 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c3955-f1f6-4f1f-9239-f95476e2936a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:56.813242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 256/500] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:57:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:56.814495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4aea7-b268-44dd-bf67-f578bc93738d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:56.814799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4aea7-b268-44dd-bf67-f578bc93738d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:56.815082 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4aea7-b268-44dd-bf67-f578bc93738d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:56.833994 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f8d4aea7-b268-44dd-bf67-f578bc93738d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:56.835192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f8d4aea7-b268-44dd-bf67-f578bc93738d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:56.846852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 245/501] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:56 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:56.848104 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737540a-3ec8-455f-86d6-c29154e579a0 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:56.848388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737540a-3ec8-455f-86d6-c29154e579a0 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:56.848659 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737540a-3ec8-455f-86d6-c29154e579a0 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:56.853279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 246/502] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:57:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:57:56.853279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737540a-3ec8-455f-86d6-c29154e579a0 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:56.896551 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:56.908882 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 257/503] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:56.909697 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4aea7-b268-44dd-bf67-f578bc93738d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:56.968404 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737540a-3ec8-455f-86d6-c29154e579a0 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:56.975677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:57.004115 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a06da9-1015-4789-9ad7-0b08977311e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:57.004734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a06da9-1015-4789-9ad7-0b08977311e2 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:57.005006 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a06da9-1015-4789-9ad7-0b08977311e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:57.005309 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a06da9-1015-4789-9ad7-0b08977311e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:57.005993 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a06da9-1015-4789-9ad7-0b08977311e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:57.006299 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a06da9-1015-4789-9ad7-0b08977311e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:57.007692 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a06da9-1015-4789-9ad7-0b08977311e2 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:57.010505 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4aea7-b268-44dd-bf67-f578bc93738d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:57.019177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 247/504] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:57:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:57.019978 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a06da9-1015-4789-9ad7-0b08977311e2 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:57.049473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6327843-a246-4e92-bb82-b6392d1ff006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:57.050123 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6327843-a246-4e92-bb82-b6392d1ff006 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:57.050407 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6327843-a246-4e92-bb82-b6392d1ff006 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:57.050688 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6327843-a246-4e92-bb82-b6392d1ff006 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:57.058319 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6327843-a246-4e92-bb82-b6392d1ff006 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:57.059750 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6327843-a246-4e92-bb82-b6392d1ff006 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:57.072455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-f6327843-a246-4e92-bb82-b6392d1ff006 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 08:57:57.072455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:57.072455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:57.072455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:57.072455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:57.072455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:57.072455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:57.072455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:57.072455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:57.072455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:57.072455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:57.072455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:57.072455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:57:57.072455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:57:57.072455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:57:57.072455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:57:57.073401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:57.073401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:57.073401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:57:57.073401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:57:57.073401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:57:57.073401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:57:57.073401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:57.073401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:57.073401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:57.073401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:57.073401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:57.073401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:57.073401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:57.073401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:57.073401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:57.073401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:57.073401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:57.073401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:57.074326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:57.074326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:57.074326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:57:57.074326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:57:57.074326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:57:57.074326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:57:57.074326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:57:57.074326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:57:57.074326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 08:57:57.074326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:57.074326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 258/505] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:57.075547 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00491d2-8046-4778-93e2-260d74a3095f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:57.075547 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00491d2-8046-4778-93e2-260d74a3095f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:57.075547 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00491d2-8046-4778-93e2-260d74a3095f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:57.092658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d00491d2-8046-4778-93e2-260d74a3095f None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:57.093860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d00491d2-8046-4778-93e2-260d74a3095f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:57.123678 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a06da9-1015-4789-9ad7-0b08977311e2 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:57.140925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:57.156931 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c985f-2fbe-4a34-88e9-fc7665e0d954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:57.157594 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c985f-2fbe-4a34-88e9-fc7665e0d954 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:57.157860 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c985f-2fbe-4a34-88e9-fc7665e0d954 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:57.158133 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c985f-2fbe-4a34-88e9-fc7665e0d954 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:57.160452 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 259/506] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:57:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:57:57.161229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00491d2-8046-4778-93e2-260d74a3095f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:57.169179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c985f-2fbe-4a34-88e9-fc7665e0d954 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:57.170646 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c985f-2fbe-4a34-88e9-fc7665e0d954 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:57.186529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-973c985f-2fbe-4a34-88e9-fc7665e0d954 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:57.186529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:57.186529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:57.186529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:57.186529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:57.186529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:57.186529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:57.186529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:57.186529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:57.186529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:57.186529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:57.186529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:57.186529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:57:57.186529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:57:57.186529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:57:57.187526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:57:57.187526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:57.187526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:57.187526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:57.187526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:57.187526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:57.187526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:57.187526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:57.187526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:57.187526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:57.187526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:57.187526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:57.187526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:57.187526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:57.187526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:57.187526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:57.187526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:57.188444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:57:57.188444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:57:57.188444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:57.188444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:57.188444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 248/507] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:57.188700 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c985f-2fbe-4a34-88e9-fc7665e0d954 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:57.208348 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:57.265901 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00491d2-8046-4778-93e2-260d74a3095f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:57.288336 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c985f-2fbe-4a34-88e9-fc7665e0d954 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:57.300174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0bac5-77eb-4a88-a773-b592170055d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:57.300817 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0bac5-77eb-4a88-a773-b592170055d4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:57.301099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0bac5-77eb-4a88-a773-b592170055d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:57.301414 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0bac5-77eb-4a88-a773-b592170055d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:57.312065 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0bac5-77eb-4a88-a773-b592170055d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:57:57.317118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-2ba0bac5-77eb-4a88-a773-b592170055d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:57:57.317118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:57.317118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:57.317118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:57.317118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:57.317118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:57.317118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:57.317118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:57.317118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:57.317118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:57.317118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:57.317118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:57.317118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:57:57.317118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:57:57.317118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:57:57.317118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:57:57.318007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:57:57.318007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:57:57.318007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:57:57.318007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:57:57.318007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:57.318007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:57.318007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:57.318007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:57.318007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:57.318007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:57.318007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:57.318007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:57.318007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:57.318007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:57.318007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:57.318007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:57.318816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:57.318816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:57.318816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:57.318816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:57.318816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:57:57.318816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:57:57.318816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:57:57.318816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:57:57.318816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:57:57.318816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:57:57.318816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:57:57.318816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:57.318816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 260/508] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:57:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:57.319492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0bac5-77eb-4a88-a773-b592170055d4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:57.327359 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91898a-0826-400f-8a4b-2dcaa3a568ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:57.328006 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91898a-0826-400f-8a4b-2dcaa3a568ba None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:57.328291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91898a-0826-400f-8a4b-2dcaa3a568ba None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:57.328567 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91898a-0826-400f-8a4b-2dcaa3a568ba None admin] PATH_INFO: `/v3/users/glance` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:57.336266 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91898a-0826-400f-8a4b-2dcaa3a568ba None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:57.338377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91898a-0826-400f-8a4b-2dcaa3a568ba None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:57.349970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-7f91898a-0826-400f-8a4b-2dcaa3a568ba None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 02 08:57:57.349970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:57.349970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:57.349970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:57.349970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:57.349970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:57.349970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:57.349970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:57.349970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:57.349970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:57.349970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:57.349970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:57.349970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:57:57.349970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:57:57.349970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:57:57.349970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:57:57.350879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:57.350879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:57.350879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:57:57.350879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:57:57.350879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:57:57.350879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:57:57.350879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:57.350879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:57.350879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:57.350879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:57.350879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:57.350879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:57.350879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:57.350879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:57.350879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:57.350879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:57.350879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:57.350879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:57.351758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:57.351758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:57.351758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:57:57.351758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:57:57.351758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:57:57.351758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:57:57.351758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:57:57.351758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:57:57.351758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 02 08:57:57.351758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:57.351758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 249/509] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:57:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:57.352632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91898a-0826-400f-8a4b-2dcaa3a568ba None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:57.365293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:57.424114 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0bac5-77eb-4a88-a773-b592170055d4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:57.431212 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:57.452896 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91898a-0826-400f-8a4b-2dcaa3a568ba None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:57.457325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf3a33-82a4-47f3-ab4b-c3fb236f262e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:57.457947 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf3a33-82a4-47f3-ab4b-c3fb236f262e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:57.458233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf3a33-82a4-47f3-ab4b-c3fb236f262e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:57.458510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf3a33-82a4-47f3-ab4b-c3fb236f262e None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:57.459299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf3a33-82a4-47f3-ab4b-c3fb236f262e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:57.459612 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf3a33-82a4-47f3-ab4b-c3fb236f262e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:57.461039 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf3a33-82a4-47f3-ab4b-c3fb236f262e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:57.479591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 261/510] 10.222.0.100 () {56 vars in 1142 bytes} [Tue Aug 2 08:57:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:57.479591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf3a33-82a4-47f3-ab4b-c3fb236f262e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:57.486906 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69605b0-7194-40f5-869e-aeb51622d81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:57.487546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69605b0-7194-40f5-869e-aeb51622d81f None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:57.487829 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69605b0-7194-40f5-869e-aeb51622d81f None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:57.488106 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69605b0-7194-40f5-869e-aeb51622d81f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:57.499179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69605b0-7194-40f5-869e-aeb51622d81f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:57.500628 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69605b0-7194-40f5-869e-aeb51622d81f None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:57.515585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-c69605b0-7194-40f5-869e-aeb51622d81f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:57.515585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:57.515585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:57.515585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:57.515585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:57.515585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:57.515585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:57.515585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:57.515585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:57.515585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:57.515585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:57.515585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:57.515585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:57:57.515585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:57:57.515585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:57:57.516446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:57:57.516446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:57.516446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:57.516446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:57.516446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:57.516446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:57.516446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:57.516446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:57.516446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:57.516446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:57.516446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:57.516446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:57.516446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:57.516446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:57.516446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:57.516446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:57.516446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:57.517306 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:57:57.517306 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:57:57.517306 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:57.517306 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:57.517306 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 250/511] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:57.517608 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69605b0-7194-40f5-869e-aeb51622d81f None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:57.574252 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:57.577287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf3a33-82a4-47f3-ab4b-c3fb236f262e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:57.611755 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a54d78-7e0e-47e2-84d1-1589b4db8981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:57.612366 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a54d78-7e0e-47e2-84d1-1589b4db8981 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:57.612648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a54d78-7e0e-47e2-84d1-1589b4db8981 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:57.612922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a54d78-7e0e-47e2-84d1-1589b4db8981 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:57.613633 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a54d78-7e0e-47e2-84d1-1589b4db8981 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:57.613940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a54d78-7e0e-47e2-84d1-1589b4db8981 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:57.615299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a54d78-7e0e-47e2-84d1-1589b4db8981 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:57.620148 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69605b0-7194-40f5-869e-aeb51622d81f None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:57.627187 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 262/512] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:57:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:57.627765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a54d78-7e0e-47e2-84d1-1589b4db8981 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:57.649991 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:57.656281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52059c69-9dfc-4743-ba5c-40e75a34b466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:57.656896 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52059c69-9dfc-4743-ba5c-40e75a34b466 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:57.657210 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52059c69-9dfc-4743-ba5c-40e75a34b466 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:57.657489 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52059c69-9dfc-4743-ba5c-40e75a34b466 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:57.658199 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52059c69-9dfc-4743-ba5c-40e75a34b466 None admin] RBAC: Adding query filter params (name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:57.658505 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52059c69-9dfc-4743-ba5c-40e75a34b466 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:57.659883 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52059c69-9dfc-4743-ba5c-40e75a34b466 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:57.668344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 251/513] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 08:57:57 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:57.669154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52059c69-9dfc-4743-ba5c-40e75a34b466 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:57.726960 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a54d78-7e0e-47e2-84d1-1589b4db8981 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:57.760480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce895a0-020b-4ca3-9399-809a6b4e2550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:57.761097 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce895a0-020b-4ca3-9399-809a6b4e2550 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:57.761407 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce895a0-020b-4ca3-9399-809a6b4e2550 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:57.761685 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce895a0-020b-4ca3-9399-809a6b4e2550 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:57.762578 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce895a0-020b-4ca3-9399-809a6b4e2550 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:57.762770 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce895a0-020b-4ca3-9399-809a6b4e2550 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:57.764187 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce895a0-020b-4ca3-9399-809a6b4e2550 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:57.771609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52059c69-9dfc-4743-ba5c-40e75a34b466 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:57.781727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 263/514] 10.222.0.100 () {56 vars in 1142 bytes} [Tue Aug 2 08:57:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:57.782567 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce895a0-020b-4ca3-9399-809a6b4e2550 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:57.801218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:57.805713 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96613eb6-d0f3-4d1e-9b43-6cc8e97ca2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:57.806841 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96613eb6-d0f3-4d1e-9b43-6cc8e97ca2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:57.807118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96613eb6-d0f3-4d1e-9b43-6cc8e97ca2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:57.807388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96613eb6-d0f3-4d1e-9b43-6cc8e97ca2a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:57.863215 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:57.873989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96613eb6-d0f3-4d1e-9b43-6cc8e97ca2a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:57.875461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96613eb6-d0f3-4d1e-9b43-6cc8e97ca2a2 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:57.884259 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce895a0-020b-4ca3-9399-809a6b4e2550 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:57.920773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674693e-a44d-4a32-99ae-5ad984a57c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:57.921433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674693e-a44d-4a32-99ae-5ad984a57c61 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:57.921719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674693e-a44d-4a32-99ae-5ad984a57c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:57.921998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674693e-a44d-4a32-99ae-5ad984a57c61 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:57.922692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674693e-a44d-4a32-99ae-5ad984a57c61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:57.923005 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674693e-a44d-4a32-99ae-5ad984a57c61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:57.924365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674693e-a44d-4a32-99ae-5ad984a57c61 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:57.936343 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 264/515] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:57:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:57.937163 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674693e-a44d-4a32-99ae-5ad984a57c61 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:57.978305 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 252/516] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:57 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:57.979045 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96613eb6-d0f3-4d1e-9b43-6cc8e97ca2a2 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:58.014470 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:58.036503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674693e-a44d-4a32-99ae-5ad984a57c61 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:58.067857 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:58.069616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937cb52-884c-4189-9381-64bb95834ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:58.070303 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937cb52-884c-4189-9381-64bb95834ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:58.070623 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937cb52-884c-4189-9381-64bb95834ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:58.070909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937cb52-884c-4189-9381-64bb95834ce7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:58.077155 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96613eb6-d0f3-4d1e-9b43-6cc8e97ca2a2 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:58.077840 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937cb52-884c-4189-9381-64bb95834ce7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:58.079273 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937cb52-884c-4189-9381-64bb95834ce7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:58.091607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-1937cb52-884c-4189-9381-64bb95834ce7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:57:58.091607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:58.091607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:58.091607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:58.091607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:58.091607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:58.091607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:58.091607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:58.091607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:58.091607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:58.091607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:58.091607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:58.091607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:57:58.091607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:57:58.091607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:57:58.092616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:57:58.092616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:58.092616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:58.092616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:58.092616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:58.092616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:58.092616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:58.092616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:58.092616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:58.092616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:58.092616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:58.092616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:58.092616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:58.092616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:58.092616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:58.092616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:58.092616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:58.093985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:57:58.093985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:57:58.093985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:57:58.093985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:57:58.093985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:57:58.093985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:57:58.093985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:57:58.093985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:58.093985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 265/517] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:57:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:58.093985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937cb52-884c-4189-9381-64bb95834ce7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:58.115155 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a71514-3465-43d7-8f34-c95c90c8424d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:58.115784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a71514-3465-43d7-8f34-c95c90c8424d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:58.116062 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a71514-3465-43d7-8f34-c95c90c8424d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:58.116331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a71514-3465-43d7-8f34-c95c90c8424d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:58.126276 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a71514-3465-43d7-8f34-c95c90c8424d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:58.127707 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a71514-3465-43d7-8f34-c95c90c8424d None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:58.142418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-65a71514-3465-43d7-8f34-c95c90c8424d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:58.142418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:58.142418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:58.142418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:58.142418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:58.142418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:58.142418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:58.142418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:58.142418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:58.142418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:58.142418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:58.142418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:58.142418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:57:58.142418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:57:58.142418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:57:58.143317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:57:58.143317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:58.143317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:58.143317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:58.143317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:58.143317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:58.143317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:58.143317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:58.143317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:58.143317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:58.143317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:58.143317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:58.143317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:58.143317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:58.143317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:58.143317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:58.143317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:58.144177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:57:58.144177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:57:58.144177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:58.144177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:58.144177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 253/518] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:58.144438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a71514-3465-43d7-8f34-c95c90c8424d None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:58.202391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937cb52-884c-4189-9381-64bb95834ce7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:58.237146 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fc4f4-891e-4e6a-b62a-05e35f892dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:58.237707 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fc4f4-891e-4e6a-b62a-05e35f892dce None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:58.237990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fc4f4-891e-4e6a-b62a-05e35f892dce None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:58.238262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fc4f4-891e-4e6a-b62a-05e35f892dce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:58.246299 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a71514-3465-43d7-8f34-c95c90c8424d None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:58.247247 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:58.281248 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c689a6-eaa5-4d73-9de4-aefecb563ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:58.281863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c689a6-eaa5-4d73-9de4-aefecb563ff4 None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:58.282157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c689a6-eaa5-4d73-9de4-aefecb563ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:58.282435 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c689a6-eaa5-4d73-9de4-aefecb563ff4 None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:58.283108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c689a6-eaa5-4d73-9de4-aefecb563ff4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:58.284548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c689a6-eaa5-4d73-9de4-aefecb563ff4 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:58.301217 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:58.316625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 254/519] 10.222.0.100 () {60 vars in 1137 bytes} [Tue Aug 2 08:57:58 2022] POST /identity/v3/users => generated 248 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:57:58.316625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c689a6-eaa5-4d73-9de4-aefecb563ff4 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:58.316909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fc4f4-891e-4e6a-b62a-05e35f892dce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:58.316909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fc4f4-891e-4e6a-b62a-05e35f892dce None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:58.431867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 266/520] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:58 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:58.432881 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fc4f4-891e-4e6a-b62a-05e35f892dce None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:58.447225 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c689a6-eaa5-4d73-9de4-aefecb563ff4 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:58.484980 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db3883-d504-49c9-8aa7-ef05f28043e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:58.485653 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db3883-d504-49c9-8aa7-ef05f28043e5 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:58.485933 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db3883-d504-49c9-8aa7-ef05f28043e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:58.486205 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db3883-d504-49c9-8aa7-ef05f28043e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:58.486776 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:58.505070 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db3883-d504-49c9-8aa7-ef05f28043e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:58.506573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db3883-d504-49c9-8aa7-ef05f28043e5 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:58.522831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-48db3883-d504-49c9-8aa7-ef05f28043e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:58.522831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:58.522831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:58.522831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:58.522831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:58.522831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:58.522831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:58.522831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:58.522831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:58.522831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:58.522831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:58.522831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:58.522831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:57:58.522831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:57:58.522831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:57:58.523844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:57:58.523844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:58.523844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:58.523844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:58.523844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:58.523844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:58.523844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:58.523844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:58.523844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:58.523844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:58.523844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:58.523844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:58.523844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:58.523844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:58.523844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:58.523844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:58.523844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:58.525490 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:57:58.525490 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:57:58.525490 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:58.525490 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:58.525490 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 255/521] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:58.525803 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db3883-d504-49c9-8aa7-ef05f28043e5 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:58.558962 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fc4f4-891e-4e6a-b62a-05e35f892dce None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:58.589182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:58.597492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5805120-6e3c-400a-92f4-e2c32694a531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:58.598198 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5805120-6e3c-400a-92f4-e2c32694a531 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:58.598841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5805120-6e3c-400a-92f4-e2c32694a531 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:58.598841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5805120-6e3c-400a-92f4-e2c32694a531 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:58.599567 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5805120-6e3c-400a-92f4-e2c32694a531 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:58.599879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5805120-6e3c-400a-92f4-e2c32694a531 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:58.601326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5805120-6e3c-400a-92f4-e2c32694a531 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:58.625437 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 267/522] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:57:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:58.625437 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5805120-6e3c-400a-92f4-e2c32694a531 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:58.644154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db3883-d504-49c9-8aa7-ef05f28043e5 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:58.679994 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c291a-f394-4d27-983d-2df74a2017b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:58.680679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c291a-f394-4d27-983d-2df74a2017b8 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:58.681100 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c291a-f394-4d27-983d-2df74a2017b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:58.681487 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c291a-f394-4d27-983d-2df74a2017b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:58.682171 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c291a-f394-4d27-983d-2df74a2017b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:58.682477 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c291a-f394-4d27-983d-2df74a2017b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:58.683830 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c291a-f394-4d27-983d-2df74a2017b8 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:58.695576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 256/523] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:57:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:58.696935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c291a-f394-4d27-983d-2df74a2017b8 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:58.734115 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:58.738631 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5805120-6e3c-400a-92f4-e2c32694a531 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:58.775775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cd173-3e7f-4617-925d-4bc263e2921c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:58.776460 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cd173-3e7f-4617-925d-4bc263e2921c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:58.776812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cd173-3e7f-4617-925d-4bc263e2921c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:58.777187 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cd173-3e7f-4617-925d-4bc263e2921c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:58.788257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43cd173-3e7f-4617-925d-4bc263e2921c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:58.789832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43cd173-3e7f-4617-925d-4bc263e2921c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:58.799383 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c291a-f394-4d27-983d-2df74a2017b8 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:58.804939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-c43cd173-3e7f-4617-925d-4bc263e2921c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:58.804939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:58.804939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:58.804939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:58.804939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:58.804939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:58.804939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:58.804939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:58.804939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:58.804939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:58.804939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:58.804939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:58.804939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:57:58.804939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:57:58.804939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:57:58.806614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:57:58.806614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:58.806614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:58.806614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:58.806614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:58.806614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:58.806614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:58.806614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:58.806614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:58.806614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:58.806614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:58.806614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:58.806614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:58.806614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:58.806614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:58.806614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:58.806614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:58.807648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:57:58.807648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:57:58.807648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:57:58.807648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:57:58.807648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 268/524] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:58.807648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cd173-3e7f-4617-925d-4bc263e2921c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:58.815282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:58.839537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677612e-0325-402c-bac4-1598aa45387a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:58.840276 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677612e-0325-402c-bac4-1598aa45387a None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:58.840609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677612e-0325-402c-bac4-1598aa45387a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:58.840893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677612e-0325-402c-bac4-1598aa45387a None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:58.841622 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b677612e-0325-402c-bac4-1598aa45387a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:58.841934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b677612e-0325-402c-bac4-1598aa45387a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:58.843596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b677612e-0325-402c-bac4-1598aa45387a None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:58.857743 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 257/525] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:57:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:58.858569 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677612e-0325-402c-bac4-1598aa45387a None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:58.926633 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cd173-3e7f-4617-925d-4bc263e2921c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:58.964189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296d762-9e64-4544-855c-3240ae99c751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:58.964815 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296d762-9e64-4544-855c-3240ae99c751 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:58.965099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296d762-9e64-4544-855c-3240ae99c751 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:58.965405 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296d762-9e64-4544-855c-3240ae99c751 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:58.966193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296d762-9e64-4544-855c-3240ae99c751 None admin] RBAC: Adding query filter params (role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=49a0c1a4a6cf43e28ec1b8488c681848) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:58.966505 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296d762-9e64-4544-855c-3240ae99c751 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:58.968111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296d762-9e64-4544-855c-3240ae99c751 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:58.970874 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677612e-0325-402c-bac4-1598aa45387a None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:58.973577 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 269/526] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:58 2022] GET /identity/v3/role_assignments?user.id=49a0c1a4a6cf43e28ec1b8488c681848&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:58.974509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296d762-9e64-4544-855c-3240ae99c751 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:59.005027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:59.037556 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64692d90-dcfe-4a14-9a80-60035a41f464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:59.038665 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64692d90-dcfe-4a14-9a80-60035a41f464 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:59.038951 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64692d90-dcfe-4a14-9a80-60035a41f464 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:59.039222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64692d90-dcfe-4a14-9a80-60035a41f464 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:59.050561 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64692d90-dcfe-4a14-9a80-60035a41f464 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:59.052803 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64692d90-dcfe-4a14-9a80-60035a41f464 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:59.070413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-64692d90-dcfe-4a14-9a80-60035a41f464 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 08:57:59.070413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:59.070413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:59.070413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:59.070413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:59.070413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:59.070413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:59.070413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:59.070413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:59.070413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:59.070413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:59.070413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:59.070413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:57:59.070413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:57:59.070413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:57:59.070413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:57:59.072187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:59.072187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:59.072187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:57:59.072187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:57:59.072187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:57:59.072187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:57:59.072187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:59.072187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:59.072187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:59.072187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:59.072187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:59.072187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:59.072187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:59.072187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:59.072187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:59.072187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:59.072187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:59.072187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:59.073640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:59.073640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:59.073640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:57:59.073640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:57:59.073640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:57:59.073640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:57:59.073640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:57:59.073640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:57:59.073640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 08:57:59.073640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:59.073640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 258/527] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:57:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:59.074784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64692d90-dcfe-4a14-9a80-60035a41f464 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:59.113397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296d762-9e64-4544-855c-3240ae99c751 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:59.118669 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:59.160959 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6af12-977d-4ffe-be13-92fe19b2044e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:59.161868 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6af12-977d-4ffe-be13-92fe19b2044e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:59.162367 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6af12-977d-4ffe-be13-92fe19b2044e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:59.162847 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6af12-977d-4ffe-be13-92fe19b2044e None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:59.163772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6af12-977d-4ffe-be13-92fe19b2044e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:59.164283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6af12-977d-4ffe-be13-92fe19b2044e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:59.167402 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6af12-977d-4ffe-be13-92fe19b2044e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:59.182996 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 270/528] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:57:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:59.183838 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6af12-977d-4ffe-be13-92fe19b2044e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:59.212576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64692d90-dcfe-4a14-9a80-60035a41f464 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:59.260238 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3518962-cfe4-463d-a7a1-7daf3eece559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:59.261488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3518962-cfe4-463d-a7a1-7daf3eece559 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:59.262574 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3518962-cfe4-463d-a7a1-7daf3eece559 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:59.263062 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3518962-cfe4-463d-a7a1-7daf3eece559 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:59.271401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:59.273077 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3518962-cfe4-463d-a7a1-7daf3eece559 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:59.276231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3518962-cfe4-463d-a7a1-7daf3eece559 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:59.294138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-a3518962-cfe4-463d-a7a1-7daf3eece559 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:57:59.294138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:59.294138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:59.294138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:59.294138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:59.294138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:59.294138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:59.294138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:59.294138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:59.294138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:59.294138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:59.294138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:59.294138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:57:59.294138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:57:59.294138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:57:59.295035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:57:59.295035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:59.295035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:59.295035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:59.295035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:59.295035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:59.295035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:59.295035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:59.295035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:59.295035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:59.295035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:59.295035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:59.295035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:59.295035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:59.295035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:59.295035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:59.295035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:59.295886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:57:59.295886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:57:59.295886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:57:59.295886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:57:59.295886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:57:59.295886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:57:59.295886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:57:59.295886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:59.296298 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 259/529] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:57:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:59.297183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3518962-cfe4-463d-a7a1-7daf3eece559 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:59.303955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6af12-977d-4ffe-be13-92fe19b2044e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:59.349867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48360a1c-5d1a-4f8d-808a-485064ae69e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:59.350845 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48360a1c-5d1a-4f8d-808a-485064ae69e9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:59.351867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48360a1c-5d1a-4f8d-808a-485064ae69e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:59.352357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48360a1c-5d1a-4f8d-808a-485064ae69e9 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:59.354259 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48360a1c-5d1a-4f8d-808a-485064ae69e9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:59.354791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48360a1c-5d1a-4f8d-808a-485064ae69e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:59.357148 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48360a1c-5d1a-4f8d-808a-485064ae69e9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:59.387822 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 271/530] 10.222.0.100 () {56 vars in 1144 bytes} [Tue Aug 2 08:57:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:59.387822 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48360a1c-5d1a-4f8d-808a-485064ae69e9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:59.412719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:59.426209 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3518962-cfe4-463d-a7a1-7daf3eece559 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:59.464728 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f941c-d15d-43a4-9844-d7c3fda4001e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:59.465434 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f941c-d15d-43a4-9844-d7c3fda4001e None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:59.465713 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f941c-d15d-43a4-9844-d7c3fda4001e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:59.466342 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f941c-d15d-43a4-9844-d7c3fda4001e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:59.474781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f941c-d15d-43a4-9844-d7c3fda4001e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:59.476233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f941c-d15d-43a4-9844-d7c3fda4001e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:59.489165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-731f941c-d15d-43a4-9844-d7c3fda4001e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:57:59.489165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:57:59.489165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:57:59.489165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:57:59.489165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:57:59.489165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:57:59.489165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:57:59.489165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:57:59.489165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:57:59.489165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:57:59.489165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:57:59.489165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:57:59.489165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:57:59.489165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:57:59.489165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:57:59.490113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:57:59.490113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:57:59.490113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:57:59.490113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:57:59.490113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:57:59.490113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:57:59.490113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:57:59.490113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:57:59.490113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:57:59.490113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:57:59.490113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:57:59.490113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:57:59.490113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:57:59.490113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:57:59.490113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:57:59.490113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:57:59.490113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:57:59.490967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:57:59.490967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:57:59.490967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:57:59.490967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:57:59.490967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:57:59.490967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:57:59.490967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:57:59.490967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:57:59.490967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 260/531] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:57:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:57:59.491696 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f941c-d15d-43a4-9844-d7c3fda4001e None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:59.523375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48360a1c-5d1a-4f8d-808a-485064ae69e9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:59.562895 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:59.568200 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c13d6-0291-4b40-9c03-9b2e1fb45c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:59.568985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c13d6-0291-4b40-9c03-9b2e1fb45c1e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:59.569447 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c13d6-0291-4b40-9c03-9b2e1fb45c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:59.569738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c13d6-0291-4b40-9c03-9b2e1fb45c1e None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:59.570508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17c13d6-0291-4b40-9c03-9b2e1fb45c1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:59.570824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17c13d6-0291-4b40-9c03-9b2e1fb45c1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:59.574128 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17c13d6-0291-4b40-9c03-9b2e1fb45c1e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:59.593161 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 272/532] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:57:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:59.593161 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c13d6-0291-4b40-9c03-9b2e1fb45c1e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:59.615009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f941c-d15d-43a4-9844-d7c3fda4001e None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:59.653323 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274785dd-5db4-4c8e-9226-7154c9501ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:59.653949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274785dd-5db4-4c8e-9226-7154c9501ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:59.654231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274785dd-5db4-4c8e-9226-7154c9501ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:59.654543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274785dd-5db4-4c8e-9226-7154c9501ac7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:59.677534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:59.715280 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c13d6-0291-4b40-9c03-9b2e1fb45c1e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:59.742322 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274785dd-5db4-4c8e-9226-7154c9501ac7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:59.744987 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274785dd-5db4-4c8e-9226-7154c9501ac7 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:59.750759 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fb085-7b07-435c-9ea5-b36f737508f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:59.751716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fb085-7b07-435c-9ea5-b36f737508f4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:59.752073 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fb085-7b07-435c-9ea5-b36f737508f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:59.752470 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fb085-7b07-435c-9ea5-b36f737508f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:59.753338 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fb085-7b07-435c-9ea5-b36f737508f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:59.753759 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fb085-7b07-435c-9ea5-b36f737508f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:59.755295 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fb085-7b07-435c-9ea5-b36f737508f4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:59.769006 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 273/533] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:57:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:59.770059 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fb085-7b07-435c-9ea5-b36f737508f4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:59.825724 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:59.860038 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 261/534] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:57:59 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:57:59.861228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274785dd-5db4-4c8e-9226-7154c9501ac7 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:59.875475 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fb085-7b07-435c-9ea5-b36f737508f4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:59.910920 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a1238-6ab8-484e-b800-591c0a2c68b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:59.911654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a1238-6ab8-484e-b800-591c0a2c68b8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:59.912299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a1238-6ab8-484e-b800-591c0a2c68b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:59.912299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a1238-6ab8-484e-b800-591c0a2c68b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:59.913257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a1238-6ab8-484e-b800-591c0a2c68b8 None admin] RBAC: Adding query filter params (role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=8e55bb6b624c45a3a08306403570177d) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:57:59.913667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a1238-6ab8-484e-b800-591c0a2c68b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:59.918092 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a1238-6ab8-484e-b800-591c0a2c68b8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:59.921781 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 274/535] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:59 2022] GET /identity/v3/role_assignments?user.id=8e55bb6b624c45a3a08306403570177d&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 260 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:57:59.925598 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:59.928049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a1238-6ab8-484e-b800-591c0a2c68b8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:59.981992 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274785dd-5db4-4c8e-9226-7154c9501ac7 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:00.029849 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eee40e-950c-4619-bf82-1d89ecebde67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:00.030831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eee40e-950c-4619-bf82-1d89ecebde67 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:00.032205 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eee40e-950c-4619-bf82-1d89ecebde67 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:00.036491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eee40e-950c-4619-bf82-1d89ecebde67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:00.037667 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eee40e-950c-4619-bf82-1d89ecebde67 None admin] RBAC: Adding query filter params (role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=d959ddd087724547a11c3d4c796d8ab2) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:00.038609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eee40e-950c-4619-bf82-1d89ecebde67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:00.040355 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eee40e-950c-4619-bf82-1d89ecebde67 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:00.054722 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 262/536] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:57:59 2022] GET /identity/v3/role_assignments?user.id=d959ddd087724547a11c3d4c796d8ab2&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:00.106037 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a1238-6ab8-484e-b800-591c0a2c68b8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:00.125545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:00.156476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dea5c8-52bc-4012-8ace-7b1608676630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:00.157332 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dea5c8-52bc-4012-8ace-7b1608676630 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:00.157699 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dea5c8-52bc-4012-8ace-7b1608676630 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:00.158056 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dea5c8-52bc-4012-8ace-7b1608676630 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:00.171949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dea5c8-52bc-4012-8ace-7b1608676630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:00.173522 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dea5c8-52bc-4012-8ace-7b1608676630 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:00.190931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-24dea5c8-52bc-4012-8ace-7b1608676630 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:00.190931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:00.190931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:00.190931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:00.190931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:00.190931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:00.190931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:00.190931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:00.190931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:00.190931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:00.190931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:00.190931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:00.190931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:00.190931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:00.190931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:00.191856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:00.191856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:00.191856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:00.191856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:00.191856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:00.191856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:00.191856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:00.191856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:00.191856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:00.191856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:00.191856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:00.191856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:00.191856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:00.191856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:00.191856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:00.191856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:00.191856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:00.192713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:00.192713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:00.192713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:00.192713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:00.192713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 275/537] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:57:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:00.198891 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eee40e-950c-4619-bf82-1d89ecebde67 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:00.311888 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eee40e-950c-4619-bf82-1d89ecebde67 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:00.330023 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:00.349590 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c58b8-23ca-4a00-abda-cf862070a01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:00.351171 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c58b8-23ca-4a00-abda-cf862070a01c None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:00.351456 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c58b8-23ca-4a00-abda-cf862070a01c None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:00.351735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c58b8-23ca-4a00-abda-cf862070a01c None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:00.352469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c58b8-23ca-4a00-abda-cf862070a01c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:00.352783 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c58b8-23ca-4a00-abda-cf862070a01c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:00.354218 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c58b8-23ca-4a00-abda-cf862070a01c None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:00.367055 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 263/538] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:00.372911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dea5c8-52bc-4012-8ace-7b1608676630 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:00.481404 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430dfc3-639b-42bb-83e9-52a90fbd0125 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:00.481742 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430dfc3-639b-42bb-83e9-52a90fbd0125 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:00.482029 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430dfc3-639b-42bb-83e9-52a90fbd0125 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:00.483664 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 264/539] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:00 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:00.489775 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63246da-61a3-48ce-bd67-60200691181e None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:00.490069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63246da-61a3-48ce-bd67-60200691181e None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:00.490409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63246da-61a3-48ce-bd67-60200691181e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:00.513486 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-b63246da-61a3-48ce-bd67-60200691181e None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:00.514908 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b63246da-61a3-48ce-bd67-60200691181e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:00.528628 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dea5c8-52bc-4012-8ace-7b1608676630 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:00.598803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a63bd-876b-4df9-b206-7cacb75c070c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:00.599467 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a63bd-876b-4df9-b206-7cacb75c070c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:00.599755 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a63bd-876b-4df9-b206-7cacb75c070c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:00.600035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a63bd-876b-4df9-b206-7cacb75c070c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:00.612104 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a63bd-876b-4df9-b206-7cacb75c070c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:00.613607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a63bd-876b-4df9-b206-7cacb75c070c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:00.632019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ff6a63bd-876b-4df9-b206-7cacb75c070c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:00.632019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:00.632019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:00.632019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:00.632019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:00.632019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:00.632019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:00.632019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:00.632019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:00.632019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:00.632019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:00.632019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:00.632019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:00.632019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:00.632019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:00.632891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:00.632891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:00.632891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:00.632891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:00.632891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:00.632891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:00.632891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:00.632891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:00.632891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:00.632891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:00.632891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:00.632891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:00.632891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:00.632891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:00.632891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:00.632891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:00.632891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:00.634553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:00.634553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:00.634553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:00.634553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:00.634553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:00.634553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:00.634553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:00.634553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:00.634553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 276/540] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:00.639891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a63bd-876b-4df9-b206-7cacb75c070c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:00.644652 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 265/541] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:00 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:58:00.669535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:00.678448 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63246da-61a3-48ce-bd67-60200691181e None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:00.810310 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a63bd-876b-4df9-b206-7cacb75c070c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:00.848194 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0324bb3-47c3-4a24-b044-5233e22dd836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:00.849387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0324bb3-47c3-4a24-b044-5233e22dd836 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:00.849878 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0324bb3-47c3-4a24-b044-5233e22dd836 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:00.850357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0324bb3-47c3-4a24-b044-5233e22dd836 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:00.851338 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0324bb3-47c3-4a24-b044-5233e22dd836 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:00.851852 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0324bb3-47c3-4a24-b044-5233e22dd836 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:00.854005 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0324bb3-47c3-4a24-b044-5233e22dd836 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:00.867329 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63246da-61a3-48ce-bd67-60200691181e None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:00.869379 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 277/542] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:00.874736 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0324bb3-47c3-4a24-b044-5233e22dd836 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:00.889962 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:00.954352 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaea8a7-bafb-4f2e-837b-184821f8dc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:00.955014 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaea8a7-bafb-4f2e-837b-184821f8dc8d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:00.955296 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaea8a7-bafb-4f2e-837b-184821f8dc8d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:00.955569 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaea8a7-bafb-4f2e-837b-184821f8dc8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:00.974985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaea8a7-bafb-4f2e-837b-184821f8dc8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:00.980902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-ecaea8a7-bafb-4f2e-837b-184821f8dc8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:00.980902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:00.980902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:00.980902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:00.980902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:00.980902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:00.980902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:00.980902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:00.980902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:00.980902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:00.980902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:00.980902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:00.980902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:00.980902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:00.980902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:00.980902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:00.981816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:00.981816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:00.981816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:00.981816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:00.981816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:00.981816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:00.981816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:00.981816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:00.981816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:00.981816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:00.981816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:00.981816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:00.981816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:00.981816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:00.981816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:00.981816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:00.982610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:00.982610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:00.982610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:00.982610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:00.982610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:00.982610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:00.982610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:00.982610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:00.982610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:00.982610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:00.982610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:00.982610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:00.989605 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 266/543] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:58:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:00.995513 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:00.995793 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaea8a7-bafb-4f2e-837b-184821f8dc8d None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:01.009693 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0324bb3-47c3-4a24-b044-5233e22dd836 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:01.059665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2cc3f-4f01-4659-9d6a-352dc8fefd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:01.060616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2cc3f-4f01-4659-9d6a-352dc8fefd98 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:01.061140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2cc3f-4f01-4659-9d6a-352dc8fefd98 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:01.061511 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2cc3f-4f01-4659-9d6a-352dc8fefd98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:01.062314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2cc3f-4f01-4659-9d6a-352dc8fefd98 None admin] RBAC: Adding query filter params (role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=f4afef33deb247509765a8c88a78ac39) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:01.062631 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2cc3f-4f01-4659-9d6a-352dc8fefd98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:01.064084 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2cc3f-4f01-4659-9d6a-352dc8fefd98 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:01.079538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 278/544] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:58:00 2022] GET /identity/v3/role_assignments?user.id=f4afef33deb247509765a8c88a78ac39&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:01.127781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaea8a7-bafb-4f2e-837b-184821f8dc8d None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:01.166237 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5817de4-ff3b-4482-991e-88ca05012a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:01.166780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5817de4-ff3b-4482-991e-88ca05012a3d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:01.167060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5817de4-ff3b-4482-991e-88ca05012a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:01.167333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5817de4-ff3b-4482-991e-88ca05012a3d None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:01.168051 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5817de4-ff3b-4482-991e-88ca05012a3d None admin] RBAC: Adding query filter params (name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:01.168358 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5817de4-ff3b-4482-991e-88ca05012a3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:01.169771 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5817de4-ff3b-4482-991e-88ca05012a3d None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:01.179873 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 267/545] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 08:58:00 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:01.185553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2cc3f-4f01-4659-9d6a-352dc8fefd98 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:01.255818 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b4c1e-f32e-4035-9a26-074af6f8a67b None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:01.256149 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b4c1e-f32e-4035-9a26-074af6f8a67b None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:01.256408 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b4c1e-f32e-4035-9a26-074af6f8a67b None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:01.257875 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 268/546] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:58:01.270110 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070852c9-6da6-4ac5-97f8-1de993c642f4 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:01.270424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070852c9-6da6-4ac5-97f8-1de993c642f4 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:01.270672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070852c9-6da6-4ac5-97f8-1de993c642f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:01.307072 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-070852c9-6da6-4ac5-97f8-1de993c642f4 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:01.308314 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-070852c9-6da6-4ac5-97f8-1de993c642f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:01.350096 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:01.364256 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2cc3f-4f01-4659-9d6a-352dc8fefd98 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:01.408907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf619ef2-16c5-4025-99e0-13f338c4f602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:01.410416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf619ef2-16c5-4025-99e0-13f338c4f602 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:01.411046 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf619ef2-16c5-4025-99e0-13f338c4f602 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:01.411565 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf619ef2-16c5-4025-99e0-13f338c4f602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:01.429199 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 269/547] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:58:01.430284 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf619ef2-16c5-4025-99e0-13f338c4f602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:01.432236 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf619ef2-16c5-4025-99e0-13f338c4f602 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:01.450035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-bf619ef2-16c5-4025-99e0-13f338c4f602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:01.450035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:01.450035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:01.450035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:01.450035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:01.450035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:01.450035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:01.450035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:01.450035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:01.450035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:01.450035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:01.450035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:01.450035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:01.450035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:01.450035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:01.450955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:01.450955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:01.450955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:01.450955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:01.450955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:01.450955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:01.450955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:01.450955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:01.450955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:01.450955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:01.450955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:01.450955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:01.450955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:01.450955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:01.450955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:01.450955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:01.450955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:01.451803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:01.451803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:01.451803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:01.451803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:01.457542 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 279/548] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:01.459856 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070852c9-6da6-4ac5-97f8-1de993c642f4 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:01.475023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf619ef2-16c5-4025-99e0-13f338c4f602 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:01.480312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:01.597266 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf619ef2-16c5-4025-99e0-13f338c4f602 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:01.619514 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070852c9-6da6-4ac5-97f8-1de993c642f4 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:01.674028 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2810fc-275c-4c35-9890-93f3c78dc76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:01.674656 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2810fc-275c-4c35-9890-93f3c78dc76b None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:01.674940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2810fc-275c-4c35-9890-93f3c78dc76b None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:01.675221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2810fc-275c-4c35-9890-93f3c78dc76b None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:01.675929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2810fc-275c-4c35-9890-93f3c78dc76b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:01.676222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2810fc-275c-4c35-9890-93f3c78dc76b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:01.677629 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2810fc-275c-4c35-9890-93f3c78dc76b None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:01.681788 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd402ecd-eee2-4db9-ba07-928ae9e2bbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:01.682387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd402ecd-eee2-4db9-ba07-928ae9e2bbcb None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:01.682670 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd402ecd-eee2-4db9-ba07-928ae9e2bbcb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:01.682945 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd402ecd-eee2-4db9-ba07-928ae9e2bbcb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:01.702308 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd402ecd-eee2-4db9-ba07-928ae9e2bbcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:01.707787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-cd402ecd-eee2-4db9-ba07-928ae9e2bbcb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:01.707787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:01.707787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:01.707787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:01.707787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:01.707787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:01.707787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:01.707787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:01.707787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:01.707787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:01.707787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:01.707787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:01.707787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:01.707787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:01.707787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:01.707787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:01.708659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:01.708659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:01.708659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:01.708659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:01.708659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:01.708659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:01.708659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:01.708659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:01.708659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:01.708659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:01.708659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:01.708659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:01.708659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:01.708659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:01.708659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:01.708659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:01.710131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:01.710131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:01.710131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:01.710131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:01.710131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:01.710131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:01.710131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:01.710131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:01.710131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:01.710131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:01.710131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:01.710131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:01.711366 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 270/549] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:01.711525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 280/550] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:58:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:01.716582 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2810fc-275c-4c35-9890-93f3c78dc76b None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:01.720178 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd402ecd-eee2-4db9-ba07-928ae9e2bbcb None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:01.753210 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:01.826506 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:01.877684 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2810fc-275c-4c35-9890-93f3c78dc76b None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:01.898500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd402ecd-eee2-4db9-ba07-928ae9e2bbcb None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:01.939925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3b1b1-e041-446b-86e5-f4e698390cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:01.940886 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3b1b1-e041-446b-86e5-f4e698390cca None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:01.941343 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3b1b1-e041-446b-86e5-f4e698390cca None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:01.941772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3b1b1-e041-446b-86e5-f4e698390cca None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:01.942922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3b1b1-e041-446b-86e5-f4e698390cca None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:01.943374 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3b1b1-e041-446b-86e5-f4e698390cca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:01.944865 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3b1b1-e041-446b-86e5-f4e698390cca None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:01.953979 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 281/551] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 08:58:01 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:01.959104 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3b1b1-e041-446b-86e5-f4e698390cca None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:02.001835 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd92c5-b7ae-4de1-86a3-e25a3bedcca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:02.002501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd92c5-b7ae-4de1-86a3-e25a3bedcca5 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:02.002781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd92c5-b7ae-4de1-86a3-e25a3bedcca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:02.003053 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd92c5-b7ae-4de1-86a3-e25a3bedcca5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:02.033701 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd92c5-b7ae-4de1-86a3-e25a3bedcca5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:02.035162 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd92c5-b7ae-4de1-86a3-e25a3bedcca5 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:02.062377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-73bd92c5-b7ae-4de1-86a3-e25a3bedcca5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:58:02.062377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:02.062377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:02.062377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:02.062377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:02.062377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:02.062377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:02.062377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:02.062377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:02.062377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:02.062377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:02.062377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:02.062377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:02.062377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:02.062377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:02.062377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:02.063451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:02.063451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:02.063451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:02.063451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:02.063451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:02.063451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:02.063451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:02.063451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:02.063451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:02.063451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:02.063451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:02.063451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:02.063451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:02.063451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:02.063451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:02.063451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:02.063451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:02.063451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:02.064492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:02.064492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:02.064492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:02.064492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:02.064492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:02.064492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:02.064492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:02.064492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:02.064492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:58:02.064492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:02.069548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 271/552] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:58:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:02.076391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3b1b1-e041-446b-86e5-f4e698390cca None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:02.078152 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd92c5-b7ae-4de1-86a3-e25a3bedcca5 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:02.110512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:02.116594 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f96d29-5c97-4479-ac04-906d10119844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:02.117620 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f96d29-5c97-4479-ac04-906d10119844 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:02.118159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f96d29-5c97-4479-ac04-906d10119844 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:02.119014 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f96d29-5c97-4479-ac04-906d10119844 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:02.131804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f96d29-5c97-4479-ac04-906d10119844 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:02.134240 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f96d29-5c97-4479-ac04-906d10119844 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:02.151586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-58f96d29-5c97-4479-ac04-906d10119844 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:02.151586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:02.151586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:02.151586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:02.151586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:02.151586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:02.151586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:02.151586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:02.151586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:02.151586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:02.151586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:02.151586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:02.151586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:02.151586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:02.151586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:02.152510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:02.152510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:02.152510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:02.152510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:02.152510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:02.152510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:02.152510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:02.152510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:02.152510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:02.152510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:02.152510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:02.152510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:02.152510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:02.152510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:02.152510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:02.152510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:02.152510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:02.153438 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:02.153438 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:02.153438 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:02.153438 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:02.155934 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 282/553] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:02.161294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f96d29-5c97-4479-ac04-906d10119844 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:02.233714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:02.263521 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd92c5-b7ae-4de1-86a3-e25a3bedcca5 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:02.307593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f723a-4732-4932-969e-efb54f78a018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:02.308603 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f723a-4732-4932-969e-efb54f78a018 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:02.309353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f723a-4732-4932-969e-efb54f78a018 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:02.310021 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f723a-4732-4932-969e-efb54f78a018 None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:02.310953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f723a-4732-4932-969e-efb54f78a018 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:02.311433 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f723a-4732-4932-969e-efb54f78a018 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:02.312067 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f96d29-5c97-4479-ac04-906d10119844 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:02.313523 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f723a-4732-4932-969e-efb54f78a018 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:02.348206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37295c73-1462-4d0d-ad27-5de9e7e5935c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:02.349007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37295c73-1462-4d0d-ad27-5de9e7e5935c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:02.349480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37295c73-1462-4d0d-ad27-5de9e7e5935c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:02.349857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37295c73-1462-4d0d-ad27-5de9e7e5935c None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:02.350664 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37295c73-1462-4d0d-ad27-5de9e7e5935c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:02.351065 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37295c73-1462-4d0d-ad27-5de9e7e5935c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:02.353404 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37295c73-1462-4d0d-ad27-5de9e7e5935c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:02.365285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 272/554] 10.222.0.100 () {56 vars in 1140 bytes} [Tue Aug 2 08:58:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:02.365285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dd8db-bf33-46e7-b997-8eca885ab3c9 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:02.365285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dd8db-bf33-46e7-b997-8eca885ab3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:02.365285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dd8db-bf33-46e7-b997-8eca885ab3c9 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:02.369185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 273/555] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:02.370176 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3dd8db-bf33-46e7-b997-8eca885ab3c9 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:02.375070 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 283/556] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:02.376713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007d0b0-e3c7-4fdc-8501-aec3ab81d327 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:02.377004 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007d0b0-e3c7-4fdc-8501-aec3ab81d327 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:02.377307 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007d0b0-e3c7-4fdc-8501-aec3ab81d327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:02.396447 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-0007d0b0-e3c7-4fdc-8501-aec3ab81d327 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:02.397749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0007d0b0-e3c7-4fdc-8501-aec3ab81d327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:02.441151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:02.491523 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3dd8db-bf33-46e7-b997-8eca885ab3c9 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:02.493620 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 284/557] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:02 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:58:02.496012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007d0b0-e3c7-4fdc-8501-aec3ab81d327 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:02.555145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90577c84-9dc4-4be9-97b9-b18a4a477343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:02.556185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90577c84-9dc4-4be9-97b9-b18a4a477343 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:02.556657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90577c84-9dc4-4be9-97b9-b18a4a477343 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:02.557164 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90577c84-9dc4-4be9-97b9-b18a4a477343 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:02.562611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:02.627707 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007d0b0-e3c7-4fdc-8501-aec3ab81d327 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:02.652884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90577c84-9dc4-4be9-97b9-b18a4a477343 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:02.655281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90577c84-9dc4-4be9-97b9-b18a4a477343 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:02.665938 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a847f-2afb-4f12-9193-726040367e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:02.666595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a847f-2afb-4f12-9193-726040367e5d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:02.667037 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a847f-2afb-4f12-9193-726040367e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:02.667330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a847f-2afb-4f12-9193-726040367e5d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:02.674626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a847f-2afb-4f12-9193-726040367e5d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:02.676062 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a847f-2afb-4f12-9193-726040367e5d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:02.687855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-b39a847f-2afb-4f12-9193-726040367e5d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:58:02.687855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:02.687855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:02.687855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:02.687855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:02.687855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:02.687855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:02.687855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:02.687855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:02.687855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:02.687855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:02.687855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:02.687855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:02.687855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:02.687855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:02.687855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:02.689305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:02.689305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:02.689305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:02.689305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:02.689305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:02.689305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:02.689305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:02.689305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:02.689305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:02.689305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:02.689305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:02.689305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:02.689305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:02.689305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:02.689305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:02.689305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:02.689305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:02.689305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:02.690245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:02.690245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:02.690245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:02.690245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:02.690245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:02.690245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:02.690245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:02.690245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:02.690245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:58:02.690245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:02.690245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 285/558] 10.222.0.100 () {56 vars in 1098 bytes} [Tue Aug 2 08:58:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:02.690245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a847f-2afb-4f12-9193-726040367e5d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:02.724978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:02.766719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 274/559] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:58:02 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:58:02.766719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b95260-9dd0-4f48-8de8-9766b14bad0f None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:02.766719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b95260-9dd0-4f48-8de8-9766b14bad0f None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:02.766719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b95260-9dd0-4f48-8de8-9766b14bad0f None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:02.766719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 275/560] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:02 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:02.766719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b95260-9dd0-4f48-8de8-9766b14bad0f None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:02.801564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:02.811042 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a847f-2afb-4f12-9193-726040367e5d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:02.845957 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de981d7c-1ec6-44dd-9c3f-02a9c2e24637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:02.847260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de981d7c-1ec6-44dd-9c3f-02a9c2e24637 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:02.847774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de981d7c-1ec6-44dd-9c3f-02a9c2e24637 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:02.848589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de981d7c-1ec6-44dd-9c3f-02a9c2e24637 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:02.860373 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de981d7c-1ec6-44dd-9c3f-02a9c2e24637 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:02.865823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-de981d7c-1ec6-44dd-9c3f-02a9c2e24637 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:02.865823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:02.865823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:02.865823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:02.865823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:02.865823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:02.865823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:02.865823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:02.865823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:02.865823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:02.865823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:02.865823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:02.865823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:02.865823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:02.865823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:02.865823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:02.866999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:02.866999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:02.866999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:02.866999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:02.866999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:02.866999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:02.866999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:02.866999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:02.866999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:02.866999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:02.866999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:02.866999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:02.866999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:02.866999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:02.866999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:02.866999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:02.867877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:02.867877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:02.867877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:02.867877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:02.867877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:02.867877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:02.867877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:02.867877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:02.867877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:02.867877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:02.867877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:02.867877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:02.867877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 286/561] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:58:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:02.867877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de981d7c-1ec6-44dd-9c3f-02a9c2e24637 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:02.882018 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b95260-9dd0-4f48-8de8-9766b14bad0f None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:02.924970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae375735-b3f0-49ee-b690-f37b16e61012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:02.925645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae375735-b3f0-49ee-b690-f37b16e61012 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:02.925922 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae375735-b3f0-49ee-b690-f37b16e61012 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:02.926189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae375735-b3f0-49ee-b690-f37b16e61012 None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:02.926974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae375735-b3f0-49ee-b690-f37b16e61012 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:02.927279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae375735-b3f0-49ee-b690-f37b16e61012 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:02.928704 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae375735-b3f0-49ee-b690-f37b16e61012 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:02.950020 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 276/562] 10.222.0.100 () {56 vars in 1138 bytes} [Tue Aug 2 08:58:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:02.951668 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83851371-6147-4430-a81b-9167fd1fcfb9 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:02.952344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83851371-6147-4430-a81b-9167fd1fcfb9 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:02.952680 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83851371-6147-4430-a81b-9167fd1fcfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:02.971206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-83851371-6147-4430-a81b-9167fd1fcfb9 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:02.972389 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-83851371-6147-4430-a81b-9167fd1fcfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:02.983314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:02.995996 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de981d7c-1ec6-44dd-9c3f-02a9c2e24637 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:03.030106 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3a999-df61-40f2-ac92-9c8861561a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:03.030689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3a999-df61-40f2-ac92-9c8861561a64 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:03.030974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3a999-df61-40f2-ac92-9c8861561a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:03.031250 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3a999-df61-40f2-ac92-9c8861561a64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:03.043808 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd3a999-df61-40f2-ac92-9c8861561a64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:03.045281 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd3a999-df61-40f2-ac92-9c8861561a64 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:03.053415 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 277/563] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:02 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:58:03.054538 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83851371-6147-4430-a81b-9167fd1fcfb9 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:03.060508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ebd3a999-df61-40f2-ac92-9c8861561a64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:03.060508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:03.060508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:03.060508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:03.060508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:03.060508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:03.060508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:03.060508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:03.060508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:03.060508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:03.060508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:03.060508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:03.060508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:03.060508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:03.060508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:03.061395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:03.061395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:03.061395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:03.061395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:03.061395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:03.061395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:03.061395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:03.061395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:03.061395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:03.061395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:03.061395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:03.061395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:03.061395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:03.061395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:03.061395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:03.061395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:03.061395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:03.062278 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:03.062278 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:03.062278 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:03.062278 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:03.062278 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 287/564] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:03.063027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3a999-df61-40f2-ac92-9c8861561a64 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:03.088623 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:03.170918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83851371-6147-4430-a81b-9167fd1fcfb9 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:03.171117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3a999-df61-40f2-ac92-9c8861561a64 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:03.206025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4f374-46c8-4420-a1d2-4e56594e5b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:03.206669 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4f374-46c8-4420-a1d2-4e56594e5b8e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:03.206949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4f374-46c8-4420-a1d2-4e56594e5b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:03.207222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4f374-46c8-4420-a1d2-4e56594e5b8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:03.207839 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fdf000-2aaa-40df-a22d-3cf7f5376e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:03.208536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdf000-2aaa-40df-a22d-3cf7f5376e24 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:03.208820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdf000-2aaa-40df-a22d-3cf7f5376e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:03.209127 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdf000-2aaa-40df-a22d-3cf7f5376e24 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:03.209840 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fdf000-2aaa-40df-a22d-3cf7f5376e24 None admin] RBAC: Adding query filter params (name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:03.210154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fdf000-2aaa-40df-a22d-3cf7f5376e24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:03.211558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fdf000-2aaa-40df-a22d-3cf7f5376e24 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:03.216403 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:03.219715 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4f374-46c8-4420-a1d2-4e56594e5b8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:03.225254 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 278/565] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 08:58:03 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:03.225396 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4f374-46c8-4420-a1d2-4e56594e5b8e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:03.225488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fdf000-2aaa-40df-a22d-3cf7f5376e24 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:03.239257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-d9b4f374-46c8-4420-a1d2-4e56594e5b8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:03.239257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:03.239257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:03.239257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:03.239257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:03.239257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:03.239257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:03.239257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:03.239257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:03.239257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:03.239257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:03.239257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:03.239257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:03.239257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:03.239257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:03.240140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:03.240140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:03.240140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:03.240140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:03.240140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:03.240140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:03.240140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:03.240140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:03.240140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:03.240140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:03.240140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:03.240140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:03.240140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:03.240140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:03.240140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:03.240140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:03.240140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:03.241045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:03.241045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:03.241045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:03.241045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:03.241045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 288/566] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:03.241948 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4f374-46c8-4420-a1d2-4e56594e5b8e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:03.319753 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:03.338481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fdf000-2aaa-40df-a22d-3cf7f5376e24 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:03.349979 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4f374-46c8-4420-a1d2-4e56594e5b8e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:03.381739 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df433e5b-4de1-41e5-a37f-c8da425c9be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:03.382366 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df433e5b-4de1-41e5-a37f-c8da425c9be1 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:03.382748 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df433e5b-4de1-41e5-a37f-c8da425c9be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:03.383126 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df433e5b-4de1-41e5-a37f-c8da425c9be1 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:03.384469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df433e5b-4de1-41e5-a37f-c8da425c9be1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:03.384780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df433e5b-4de1-41e5-a37f-c8da425c9be1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:03.386167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ea445-c10f-4f3a-8edb-e4b72ae9707f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:03.386298 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df433e5b-4de1-41e5-a37f-c8da425c9be1 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:03.386834 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ea445-c10f-4f3a-8edb-e4b72ae9707f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:03.387115 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ea445-c10f-4f3a-8edb-e4b72ae9707f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:03.387384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ea445-c10f-4f3a-8edb-e4b72ae9707f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:03.398252 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 279/567] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:03.398944 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df433e5b-4de1-41e5-a37f-c8da425c9be1 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:03.407566 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ea445-c10f-4f3a-8edb-e4b72ae9707f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:03.413127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-b72ea445-c10f-4f3a-8edb-e4b72ae9707f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:03.413127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:03.413127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:03.413127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:03.413127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:03.413127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:03.413127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:03.413127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:03.413127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:03.413127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:03.413127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:03.413127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:03.413127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:03.413127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:03.413127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:03.413127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:03.414263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:03.414263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:03.414263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:03.414263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:03.414263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:03.414263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:03.414263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:03.414263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:03.414263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:03.414263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:03.414263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:03.414263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:03.414263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:03.414263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:03.414263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:03.414263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:03.415413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:03.415413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:03.415413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:03.415413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:03.415413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:03.415413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:03.415413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:03.415413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:03.415413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:03.415413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:03.415413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:03.415413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:03.416354 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 289/568] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:58:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:03.416895 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ea445-c10f-4f3a-8edb-e4b72ae9707f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:03.473210 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:03.510349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df433e5b-4de1-41e5-a37f-c8da425c9be1 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:03.521980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ea445-c10f-4f3a-8edb-e4b72ae9707f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:03.543809 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3cb2a8-3892-47a3-bed7-4d66f12e1d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:03.544500 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3cb2a8-3892-47a3-bed7-4d66f12e1d92 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:03.544780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3cb2a8-3892-47a3-bed7-4d66f12e1d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:03.545051 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3cb2a8-3892-47a3-bed7-4d66f12e1d92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:03.556184 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3cb2a8-3892-47a3-bed7-4d66f12e1d92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:03.557356 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287b650-90f5-4a22-bf61-a178fb8c2485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:03.557685 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3cb2a8-3892-47a3-bed7-4d66f12e1d92 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:03.557963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287b650-90f5-4a22-bf61-a178fb8c2485 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:03.558244 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287b650-90f5-4a22-bf61-a178fb8c2485 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:03.558515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287b650-90f5-4a22-bf61-a178fb8c2485 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:03.559198 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287b650-90f5-4a22-bf61-a178fb8c2485 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:03.559508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287b650-90f5-4a22-bf61-a178fb8c2485 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:03.560834 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287b650-90f5-4a22-bf61-a178fb8c2485 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:03.564290 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:03.573155 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 290/569] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:03.573882 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287b650-90f5-4a22-bf61-a178fb8c2485 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:03.574351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-7c3cb2a8-3892-47a3-bed7-4d66f12e1d92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:03.574351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:03.574351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:03.574351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:03.574351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:03.574351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:03.574351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:03.574351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:03.574351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:03.574351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:03.574351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:03.574351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:03.574351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:03.574351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:03.574351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:03.575221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:03.575221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:03.575221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:03.575221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:03.575221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:03.575221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:03.575221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:03.575221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:03.575221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:03.575221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:03.575221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:03.575221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:03.575221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:03.575221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:03.575221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:03.575221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:03.575221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:03.576142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:03.576142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:03.576142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:03.576142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:03.577248 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 280/570] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:03.578097 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3cb2a8-3892-47a3-bed7-4d66f12e1d92 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:03.676261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287b650-90f5-4a22-bf61-a178fb8c2485 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:03.678102 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3cb2a8-3892-47a3-bed7-4d66f12e1d92 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:03.689544 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:03.711504 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae4583-56bd-4681-8e76-404f61b50db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:03.711790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27eac5-f147-42c4-a936-f0a396c1ab77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:03.712206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae4583-56bd-4681-8e76-404f61b50db0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:03.712403 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27eac5-f147-42c4-a936-f0a396c1ab77 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:03.712508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae4583-56bd-4681-8e76-404f61b50db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:03.712687 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27eac5-f147-42c4-a936-f0a396c1ab77 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:03.712774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae4583-56bd-4681-8e76-404f61b50db0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:03.712965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27eac5-f147-42c4-a936-f0a396c1ab77 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:03.713718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27eac5-f147-42c4-a936-f0a396c1ab77 None admin] RBAC: Adding query filter params (name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:03.714026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27eac5-f147-42c4-a936-f0a396c1ab77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:03.715446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27eac5-f147-42c4-a936-f0a396c1ab77 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:03.719397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae4583-56bd-4681-8e76-404f61b50db0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:03.720818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae4583-56bd-4681-8e76-404f61b50db0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:03.723365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 281/571] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 08:58:03 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:03.724603 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac020a-b72d-4f5e-be62-046729d47c57 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:03.724898 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac020a-b72d-4f5e-be62-046729d47c57 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:03.725201 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac020a-b72d-4f5e-be62-046729d47c57 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:03.729253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 282/572] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:03.729253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac020a-b72d-4f5e-be62-046729d47c57 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:03.735835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-89ae4583-56bd-4681-8e76-404f61b50db0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:03.735835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:03.735835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:03.735835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:03.735835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:03.735835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:03.735835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:03.735835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:03.735835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:03.735835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:03.735835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:03.735835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:03.735835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:03.735835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:03.735835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:03.736713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:03.736713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:03.736713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:03.736713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:03.736713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:03.736713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:03.736713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:03.736713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:03.736713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:03.736713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:03.736713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:03.736713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:03.736713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:03.736713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:03.736713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:03.736713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:03.736713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:03.737695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:03.737695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:03.737695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:03.737695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:03.737695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:03.737695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:03.737695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:03.737695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:03.737695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 291/573] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:03.738183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae4583-56bd-4681-8e76-404f61b50db0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:03.792622 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:03.834412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac020a-b72d-4f5e-be62-046729d47c57 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:03.838713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae4583-56bd-4681-8e76-404f61b50db0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:03.867694 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfa086-ba40-47b7-9926-838a586f14dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:03.868384 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfa086-ba40-47b7-9926-838a586f14dc None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:03.868635 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfa086-ba40-47b7-9926-838a586f14dc None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:03.868911 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfa086-ba40-47b7-9926-838a586f14dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:03.872252 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc955f20-076a-4e10-abbd-6620e9efbd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:03.873202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc955f20-076a-4e10-abbd-6620e9efbd13 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:03.873490 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc955f20-076a-4e10-abbd-6620e9efbd13 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:03.873766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc955f20-076a-4e10-abbd-6620e9efbd13 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:03.874451 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc955f20-076a-4e10-abbd-6620e9efbd13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:03.874769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc955f20-076a-4e10-abbd-6620e9efbd13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:03.876053 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfa086-ba40-47b7-9926-838a586f14dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:03.876384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc955f20-076a-4e10-abbd-6620e9efbd13 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:03.877515 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfa086-ba40-47b7-9926-838a586f14dc None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:03.888045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 292/574] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:03.888826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc955f20-076a-4e10-abbd-6620e9efbd13 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:03.889810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-11dfa086-ba40-47b7-9926-838a586f14dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:03.889810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:03.889810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:03.889810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:03.889810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:03.889810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:03.889810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:03.889810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:03.889810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:03.889810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:03.889810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:03.889810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:03.889810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:03.889810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:03.889810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:03.890673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:03.890673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:03.890673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:03.890673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:03.890673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:03.890673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:03.890673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:03.890673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:03.890673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:03.890673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:03.890673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:03.890673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:03.890673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:03.890673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:03.890673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:03.890673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:03.890673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:03.891593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:03.891593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:03.891593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:03.891593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:03.891593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:03.891593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:03.891593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:03.891593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:03.893357 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 283/575] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:03.894635 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495be47-d285-487b-93d0-ae90f112e901 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:03.894919 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495be47-d285-487b-93d0-ae90f112e901 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:03.895194 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495be47-d285-487b-93d0-ae90f112e901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:03.913435 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-6495be47-d285-487b-93d0-ae90f112e901 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:03.914620 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-6495be47-d285-487b-93d0-ae90f112e901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:03.925711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:03.982388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 284/576] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:58:03.983216 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495be47-d285-487b-93d0-ae90f112e901 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:03.997022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc955f20-076a-4e10-abbd-6620e9efbd13 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:04.030332 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164dd469-bc57-4af1-b9d1-7e0d9cfe05c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:04.030968 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164dd469-bc57-4af1-b9d1-7e0d9cfe05c9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:04.031245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164dd469-bc57-4af1-b9d1-7e0d9cfe05c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:04.031528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164dd469-bc57-4af1-b9d1-7e0d9cfe05c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:04.041978 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:04.042948 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164dd469-bc57-4af1-b9d1-7e0d9cfe05c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:04.044612 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164dd469-bc57-4af1-b9d1-7e0d9cfe05c9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:04.059054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-164dd469-bc57-4af1-b9d1-7e0d9cfe05c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:04.059054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:04.059054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:04.059054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:04.059054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:04.059054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:04.059054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:04.059054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:04.059054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:04.059054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:04.059054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:04.059054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:04.059054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:04.059054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:04.059054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:04.059980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:04.059980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:04.059980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:04.059980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:04.059980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:04.059980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:04.059980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:04.059980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:04.059980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:04.059980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:04.059980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:04.059980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:04.059980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:04.059980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:04.059980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:04.059980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:04.059980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:04.060867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:04.060867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:04.060867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:04.060867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:04.060867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 293/577] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:04.061186 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164dd469-bc57-4af1-b9d1-7e0d9cfe05c9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:04.092931 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495be47-d285-487b-93d0-ae90f112e901 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:04.128334 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26342f3-df82-4949-acd1-d676a84d52e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:04.128944 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26342f3-df82-4949-acd1-d676a84d52e3 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:04.129259 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26342f3-df82-4949-acd1-d676a84d52e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:04.129531 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26342f3-df82-4949-acd1-d676a84d52e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:04.130271 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26342f3-df82-4949-acd1-d676a84d52e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:04.130574 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26342f3-df82-4949-acd1-d676a84d52e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:04.131994 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26342f3-df82-4949-acd1-d676a84d52e3 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:04.148259 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:04.149680 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 285/578] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:04.150559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26342f3-df82-4949-acd1-d676a84d52e3 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:04.170005 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164dd469-bc57-4af1-b9d1-7e0d9cfe05c9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:04.203584 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392df083-b1f0-4965-8b44-82865b80c0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:04.204231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392df083-b1f0-4965-8b44-82865b80c0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:04.204518 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392df083-b1f0-4965-8b44-82865b80c0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:04.204793 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392df083-b1f0-4965-8b44-82865b80c0c2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:04.213828 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392df083-b1f0-4965-8b44-82865b80c0c2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:04.215270 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392df083-b1f0-4965-8b44-82865b80c0c2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:04.228548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-392df083-b1f0-4965-8b44-82865b80c0c2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 08:58:04.228548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:04.228548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:04.228548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:04.228548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:04.228548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:04.228548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:04.228548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:04.228548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:04.228548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:04.228548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:04.228548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:04.228548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:04.228548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:04.228548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:04.228548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:04.229550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:04.229550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:04.229550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:04.229550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:04.229550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:04.229550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:04.229550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:04.229550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:04.229550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:04.229550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:04.229550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:04.229550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:04.229550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:04.229550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:04.229550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:04.229550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:04.229550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:04.229550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:04.230915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:04.230915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:04.230915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:04.230915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:04.230915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:04.230915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:04.230915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:04.230915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:04.230915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 08:58:04.230915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:04.230915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 294/579] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:58:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:04.230915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392df083-b1f0-4965-8b44-82865b80c0c2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:04.255492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26342f3-df82-4949-acd1-d676a84d52e3 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:04.266636 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:04.291652 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344adf5-6bcf-4433-9f7a-26f5ade0d7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:04.292270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344adf5-6bcf-4433-9f7a-26f5ade0d7ba None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:04.292552 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344adf5-6bcf-4433-9f7a-26f5ade0d7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:04.293087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344adf5-6bcf-4433-9f7a-26f5ade0d7ba None admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:04.293621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344adf5-6bcf-4433-9f7a-26f5ade0d7ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:04.293929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344adf5-6bcf-4433-9f7a-26f5ade0d7ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:04.295367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344adf5-6bcf-4433-9f7a-26f5ade0d7ba None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:04.306633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 286/580] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:04.307416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344adf5-6bcf-4433-9f7a-26f5ade0d7ba None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:04.334346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392df083-b1f0-4965-8b44-82865b80c0c2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:04.368267 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8422d48-ce5e-4fd3-9645-b66381d63c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:04.368838 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8422d48-ce5e-4fd3-9645-b66381d63c1f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:04.369148 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8422d48-ce5e-4fd3-9645-b66381d63c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:04.369430 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8422d48-ce5e-4fd3-9645-b66381d63c1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:04.379633 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8422d48-ce5e-4fd3-9645-b66381d63c1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:04.385552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-f8422d48-ce5e-4fd3-9645-b66381d63c1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:04.385552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:04.385552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:04.385552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:04.385552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:04.385552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:04.385552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:04.385552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:04.385552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:04.385552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:04.385552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:04.385552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:04.385552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:04.385552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:04.385552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:04.385552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:04.386468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:04.386468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:04.386468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:04.386468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:04.386468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:04.386468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:04.386468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:04.386468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:04.386468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:04.386468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:04.386468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:04.386468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:04.386468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:04.386468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:04.386468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:04.386468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:04.387311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:04.387311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:04.387311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:04.387311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:04.387311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:04.387311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:04.387311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:04.387311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:04.387311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:04.387311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:04.387311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:04.387311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:04.387311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 295/581] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:58:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:04.388025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8422d48-ce5e-4fd3-9645-b66381d63c1f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:04.391200 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:04.413997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344adf5-6bcf-4433-9f7a-26f5ade0d7ba None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:04.457673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291bf3b-b7ba-4ef0-af96-1c2db987c050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:04.458384 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291bf3b-b7ba-4ef0-af96-1c2db987c050 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:04.458667 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291bf3b-b7ba-4ef0-af96-1c2db987c050 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:04.458938 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291bf3b-b7ba-4ef0-af96-1c2db987c050 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:04.459619 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7291bf3b-b7ba-4ef0-af96-1c2db987c050 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:04.460245 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7291bf3b-b7ba-4ef0-af96-1c2db987c050 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:04.461325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7291bf3b-b7ba-4ef0-af96-1c2db987c050 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:04.479756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 287/582] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:04.480746 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291bf3b-b7ba-4ef0-af96-1c2db987c050 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:04.517297 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8422d48-ce5e-4fd3-9645-b66381d63c1f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:04.518601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:04.555236 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf35f5-ab56-4157-93c4-79326fa5380a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:04.556587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf35f5-ab56-4157-93c4-79326fa5380a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:04.556875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf35f5-ab56-4157-93c4-79326fa5380a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:04.557192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf35f5-ab56-4157-93c4-79326fa5380a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:04.557989 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf35f5-ab56-4157-93c4-79326fa5380a None admin] RBAC: Adding query filter params (role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=50cb532b58ed4951ba960a0a7d579b49) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:04.558303 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf35f5-ab56-4157-93c4-79326fa5380a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:04.559733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf35f5-ab56-4157-93c4-79326fa5380a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:04.565211 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 296/583] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:58:04 2022] GET /identity/v3/role_assignments?user.id=50cb532b58ed4951ba960a0a7d579b49&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:04.565978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf35f5-ab56-4157-93c4-79326fa5380a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:04.611123 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291bf3b-b7ba-4ef0-af96-1c2db987c050 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:04.656995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a0816-ecde-4c74-a7e4-99bd0c7521bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:04.657689 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a0816-ecde-4c74-a7e4-99bd0c7521bb None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:04.658089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a0816-ecde-4c74-a7e4-99bd0c7521bb None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:04.658402 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a0816-ecde-4c74-a7e4-99bd0c7521bb None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:04.659312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a0816-ecde-4c74-a7e4-99bd0c7521bb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:04.659708 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a0816-ecde-4c74-a7e4-99bd0c7521bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:04.661327 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a0816-ecde-4c74-a7e4-99bd0c7521bb None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:04.662937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:04.690351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 288/584] 10.222.0.100 () {56 vars in 1142 bytes} [Tue Aug 2 08:58:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:04.691736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a0816-ecde-4c74-a7e4-99bd0c7521bb None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:04.710914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf35f5-ab56-4157-93c4-79326fa5380a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:04.752489 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be67b6-d427-4450-a90c-124bf8ce661a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:04.753209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be67b6-d427-4450-a90c-124bf8ce661a None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:04.753502 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be67b6-d427-4450-a90c-124bf8ce661a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:04.753783 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be67b6-d427-4450-a90c-124bf8ce661a None admin] PATH_INFO: `/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/49a0c1a4a6cf43e28ec1b8488c681848/roles/6e6ac8e8e7044b209d1a8b5bb6e4b1d8` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:04.769820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be67b6-d427-4450-a90c-124bf8ce661a None admin] RBAC: Authorizing `identity:create_grant(project_id=61ca0176324e445b94e42a5214d2c89b, user_id=49a0c1a4a6cf43e28ec1b8488c681848, role_id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:04.771416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be67b6-d427-4450-a90c-124bf8ce661a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:04.786615 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 297/585] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:58:04 2022] PUT /identity/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/49a0c1a4a6cf43e28ec1b8488c681848/roles/6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:58:04.787443 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be67b6-d427-4450-a90c-124bf8ce661a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:04.805434 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:04.820434 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a0816-ecde-4c74-a7e4-99bd0c7521bb None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:04.861662 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7aeda-c6e7-48c2-a480-202718c6df47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:04.862297 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7aeda-c6e7-48c2-a480-202718c6df47 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:04.862576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7aeda-c6e7-48c2-a480-202718c6df47 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:04.862852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7aeda-c6e7-48c2-a480-202718c6df47 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:04.863579 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7aeda-c6e7-48c2-a480-202718c6df47 None admin] RBAC: Adding query filter params (name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:04.863883 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7aeda-c6e7-48c2-a480-202718c6df47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:04.865582 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7aeda-c6e7-48c2-a480-202718c6df47 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:04.875325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 289/586] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 08:58:04 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:04.876476 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7aeda-c6e7-48c2-a480-202718c6df47 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:04.951481 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be67b6-d427-4450-a90c-124bf8ce661a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:04.976578 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:05.002905 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7aeda-c6e7-48c2-a480-202718c6df47 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:05.003872 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a650c1d7-dbfa-41aa-8219-1acc17310ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:05.004532 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650c1d7-dbfa-41aa-8219-1acc17310ccf None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:05.004828 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650c1d7-dbfa-41aa-8219-1acc17310ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:05.005092 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650c1d7-dbfa-41aa-8219-1acc17310ccf None admin] PATH_INFO: `/v3/users/glance` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:05.013308 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650c1d7-dbfa-41aa-8219-1acc17310ccf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:05.014758 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650c1d7-dbfa-41aa-8219-1acc17310ccf None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:05.030093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-a650c1d7-dbfa-41aa-8219-1acc17310ccf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 02 08:58:05.030093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:05.030093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:05.030093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:05.030093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:05.030093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:05.030093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:05.030093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:05.030093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:05.030093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:05.030093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:05.030093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:05.030093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:05.030093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:05.030093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:05.030093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:05.031304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:05.031304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:05.031304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:05.031304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:05.031304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:05.031304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:05.031304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:05.031304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:05.031304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:05.031304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:05.031304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:05.031304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:05.031304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:05.031304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:05.031304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:05.031304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:05.031304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:05.031304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:05.032655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:05.032655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:05.032655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:05.032655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:05.032655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:05.032655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:05.032655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:05.032655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:05.032655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 02 08:58:05.032655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:05.032655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 298/587] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:58:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:05.032655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a650c1d7-dbfa-41aa-8219-1acc17310ccf None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:05.046572 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534dfbd2-752a-4227-9628-a31f89c0e0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:05.047216 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dfbd2-752a-4227-9628-a31f89c0e0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:05.047492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dfbd2-752a-4227-9628-a31f89c0e0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:05.047763 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dfbd2-752a-4227-9628-a31f89c0e0f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:05.059111 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534dfbd2-752a-4227-9628-a31f89c0e0f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:05.060575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534dfbd2-752a-4227-9628-a31f89c0e0f2 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:05.077002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-534dfbd2-752a-4227-9628-a31f89c0e0f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:05.077002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:05.077002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:05.077002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:05.077002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:05.077002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:05.077002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:05.077002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:05.077002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:05.077002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:05.077002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:05.077002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:05.077002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:05.077002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:05.077002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:05.077989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:05.077989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:05.077989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:05.077989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:05.077989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:05.077989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:05.077989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:05.077989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:05.077989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:05.077989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:05.077989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:05.077989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:05.077989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:05.077989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:05.077989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:05.077989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:05.077989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:05.078897 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:05.078897 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:05.078897 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:05.078897 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:05.078897 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 290/588] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:05.079313 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534dfbd2-752a-4227-9628-a31f89c0e0f2 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:05.089457 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:05.162995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a650c1d7-dbfa-41aa-8219-1acc17310ccf None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:05.190538 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534dfbd2-752a-4227-9628-a31f89c0e0f2 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:05.202751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c71b3c-fae3-4e11-8687-8531f2c51e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:05.203668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c71b3c-fae3-4e11-8687-8531f2c51e86 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:05.204104 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c71b3c-fae3-4e11-8687-8531f2c51e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:05.204628 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c71b3c-fae3-4e11-8687-8531f2c51e86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:05.218217 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c71b3c-fae3-4e11-8687-8531f2c51e86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:05.220016 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c71b3c-fae3-4e11-8687-8531f2c51e86 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:05.227130 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4f036-d7d8-41ab-bed5-ff3a43565921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:05.228147 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4f036-d7d8-41ab-bed5-ff3a43565921 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:05.228429 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4f036-d7d8-41ab-bed5-ff3a43565921 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:05.228708 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4f036-d7d8-41ab-bed5-ff3a43565921 None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:05.229606 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e4f036-d7d8-41ab-bed5-ff3a43565921 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:05.229913 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e4f036-d7d8-41ab-bed5-ff3a43565921 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:05.231382 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e4f036-d7d8-41ab-bed5-ff3a43565921 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:05.237548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-96c71b3c-fae3-4e11-8687-8531f2c51e86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:05.237548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:05.237548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:05.237548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:05.237548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:05.237548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:05.237548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:05.237548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:05.237548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:05.237548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:05.237548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:05.237548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:05.237548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:05.237548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:05.237548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:05.238702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:05.238702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:05.238702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:05.238702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:05.238702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:05.238702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:05.238702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:05.238702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:05.238702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:05.238702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:05.238702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:05.238702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:05.238702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:05.238702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:05.238702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:05.238702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:05.238702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:05.239716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:05.239716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:05.239716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:05.239716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:05.241720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 299/589] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:05.242831 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c71b3c-fae3-4e11-8687-8531f2c51e86 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:05.255289 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 291/590] 10.222.0.100 () {56 vars in 1142 bytes} [Tue Aug 2 08:58:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:05.256193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4f036-d7d8-41ab-bed5-ff3a43565921 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:05.278381 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:05.319695 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:05.358984 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c71b3c-fae3-4e11-8687-8531f2c51e86 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:05.364631 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4f036-d7d8-41ab-bed5-ff3a43565921 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:05.395237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e720774-2924-442d-8a09-02e5ca6248e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:05.396539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e720774-2924-442d-8a09-02e5ca6248e3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:05.397025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e720774-2924-442d-8a09-02e5ca6248e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:05.397555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e720774-2924-442d-8a09-02e5ca6248e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:05.398448 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e720774-2924-442d-8a09-02e5ca6248e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:05.399140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e720774-2924-442d-8a09-02e5ca6248e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:05.401007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e720774-2924-442d-8a09-02e5ca6248e3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:05.401878 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e8f02-f8c5-4624-a14e-15fbd07e9699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:05.402561 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e8f02-f8c5-4624-a14e-15fbd07e9699 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:05.402945 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e8f02-f8c5-4624-a14e-15fbd07e9699 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:05.403353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e8f02-f8c5-4624-a14e-15fbd07e9699 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:05.404101 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e8f02-f8c5-4624-a14e-15fbd07e9699 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:05.404860 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e8f02-f8c5-4624-a14e-15fbd07e9699 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:05.405996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e8f02-f8c5-4624-a14e-15fbd07e9699 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:05.415947 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 300/591] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:05.417005 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e720774-2924-442d-8a09-02e5ca6248e3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:05.418796 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 292/592] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:05.420907 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e8f02-f8c5-4624-a14e-15fbd07e9699 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:05.511060 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:05.520754 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e720774-2924-442d-8a09-02e5ca6248e3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:05.529936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e8f02-f8c5-4624-a14e-15fbd07e9699 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:05.551885 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:05.559498 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248eeef-9de0-484e-8f16-c6aa43934b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:05.560476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248eeef-9de0-484e-8f16-c6aa43934b15 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:05.560982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248eeef-9de0-484e-8f16-c6aa43934b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:05.561489 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248eeef-9de0-484e-8f16-c6aa43934b15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:05.568014 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557457f0-9042-4d76-b133-11a96e375085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:05.568651 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557457f0-9042-4d76-b133-11a96e375085 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:05.568929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557457f0-9042-4d76-b133-11a96e375085 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:05.569243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557457f0-9042-4d76-b133-11a96e375085 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:05.575189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248eeef-9de0-484e-8f16-c6aa43934b15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:05.576868 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248eeef-9de0-484e-8f16-c6aa43934b15 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:05.577235 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557457f0-9042-4d76-b133-11a96e375085 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:05.578669 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557457f0-9042-4d76-b133-11a96e375085 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:05.591851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-557457f0-9042-4d76-b133-11a96e375085 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:05.591851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:05.591851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:05.591851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:05.591851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:05.591851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:05.591851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:05.591851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:05.591851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:05.591851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:05.591851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:05.591851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:05.591851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:05.591851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:05.591851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:05.592784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:05.592784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:05.592784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:05.592784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:05.592784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:05.592784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:05.592784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:05.592784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:05.592784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:05.592784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:05.592784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:05.592784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:05.592784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:05.592784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:05.592784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:05.592784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:05.592784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:05.593740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:05.593740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:05.593740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:05.593740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:05.593740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:05.593740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:05.593740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:05.593740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:05.593740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 293/593] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:05.594231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557457f0-9042-4d76-b133-11a96e375085 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:05.596588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-5248eeef-9de0-484e-8f16-c6aa43934b15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:05.596588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:05.596588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:05.596588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:05.596588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:05.596588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:05.596588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:05.596588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:05.596588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:05.596588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:05.596588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:05.596588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:05.596588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:05.596588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:05.596588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:05.597738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:05.597738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:05.597738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:05.597738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:05.597738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:05.597738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:05.597738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:05.597738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:05.597738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:05.597738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:05.597738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:05.597738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:05.597738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:05.597738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:05.597738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:05.597738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:05.597738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:05.598652 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:05.598652 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:05.598652 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:05.598652 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:05.600665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 301/594] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:05.601704 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248eeef-9de0-484e-8f16-c6aa43934b15 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:05.704210 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557457f0-9042-4d76-b133-11a96e375085 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:05.708243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248eeef-9de0-484e-8f16-c6aa43934b15 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:05.742961 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01de7a-5d8f-49a4-a25e-5d295f901c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:05.743607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01de7a-5d8f-49a4-a25e-5d295f901c23 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:05.743885 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01de7a-5d8f-49a4-a25e-5d295f901c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:05.744158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01de7a-5d8f-49a4-a25e-5d295f901c23 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:05.747373 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ca0b3-4eb7-4a42-bf4c-b65f523e8b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:05.748241 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ca0b3-4eb7-4a42-bf4c-b65f523e8b5f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:05.748729 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ca0b3-4eb7-4a42-bf4c-b65f523e8b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:05.749238 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ca0b3-4eb7-4a42-bf4c-b65f523e8b5f None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:05.750193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ca0b3-4eb7-4a42-bf4c-b65f523e8b5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:05.750980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ca0b3-4eb7-4a42-bf4c-b65f523e8b5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:05.752620 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ca0b3-4eb7-4a42-bf4c-b65f523e8b5f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:05.753049 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df01de7a-5d8f-49a4-a25e-5d295f901c23 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:05.754505 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df01de7a-5d8f-49a4-a25e-5d295f901c23 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:05.766424 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 302/595] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:05.766570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-df01de7a-5d8f-49a4-a25e-5d295f901c23 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 08:58:05.766570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:05.766570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:05.766570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:05.766570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:05.766570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:05.766570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:05.766570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:05.766570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:05.766570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:05.766570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:05.766570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:05.766570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:05.766570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:05.766570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:05.766570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:05.767493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:05.767493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:05.767493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:05.767493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:05.767493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:05.767493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:05.767493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:05.767493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:05.767493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:05.767493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:05.767493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:05.767493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:05.767493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:05.767493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:05.767493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:05.767493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:05.767493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:05.767493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:05.768438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:05.768438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:05.768438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:05.768438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:05.768438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:05.768438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:05.768438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:05.768438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:05.768438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 08:58:05.768438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:05.768438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 294/596] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:58:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:05.769024 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01de7a-5d8f-49a4-a25e-5d295f901c23 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:05.771924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ca0b3-4eb7-4a42-bf4c-b65f523e8b5f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:05.775450 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:05.815966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:05.878637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ca0b3-4eb7-4a42-bf4c-b65f523e8b5f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:05.879834 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01de7a-5d8f-49a4-a25e-5d295f901c23 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:05.916103 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef339a4b-8e44-456f-a9ad-b2fa7e013a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:05.917068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a9860-8817-4c4d-9ebb-ab8639c66f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:05.917380 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef339a4b-8e44-456f-a9ad-b2fa7e013a8b None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:05.917878 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef339a4b-8e44-456f-a9ad-b2fa7e013a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:05.917985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a9860-8817-4c4d-9ebb-ab8639c66f62 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:05.918237 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a9860-8817-4c4d-9ebb-ab8639c66f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:05.918601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a9860-8817-4c4d-9ebb-ab8639c66f62 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:05.918764 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef339a4b-8e44-456f-a9ad-b2fa7e013a8b None admin] PATH_INFO: `/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/8e55bb6b624c45a3a08306403570177d/roles/6e6ac8e8e7044b209d1a8b5bb6e4b1d8` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:05.919413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a9860-8817-4c4d-9ebb-ab8639c66f62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:05.919781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a9860-8817-4c4d-9ebb-ab8639c66f62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:05.921254 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a9860-8817-4c4d-9ebb-ab8639c66f62 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:05.934521 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 295/597] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:05.935501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a9860-8817-4c4d-9ebb-ab8639c66f62 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:05.936420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef339a4b-8e44-456f-a9ad-b2fa7e013a8b None admin] RBAC: Authorizing `identity:create_grant(project_id=61ca0176324e445b94e42a5214d2c89b, user_id=8e55bb6b624c45a3a08306403570177d, role_id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:05.938094 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef339a4b-8e44-456f-a9ad-b2fa7e013a8b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:05.954580 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 303/598] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:58:05 2022] PUT /identity/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/8e55bb6b624c45a3a08306403570177d/roles/6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:58:05.955785 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef339a4b-8e44-456f-a9ad-b2fa7e013a8b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:06.018888 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:06.074031 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef339a4b-8e44-456f-a9ad-b2fa7e013a8b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:06.115308 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cfbf9-b09f-41ac-aa2c-de0c07103e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:06.116277 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cfbf9-b09f-41ac-aa2c-de0c07103e91 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:06.116744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cfbf9-b09f-41ac-aa2c-de0c07103e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:06.117382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cfbf9-b09f-41ac-aa2c-de0c07103e91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:06.125496 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:06.129812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9cfbf9-b09f-41ac-aa2c-de0c07103e91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:06.131281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a9860-8817-4c4d-9ebb-ab8639c66f62 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:06.131628 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9cfbf9-b09f-41ac-aa2c-de0c07103e91 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:06.147010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-6d9cfbf9-b09f-41ac-aa2c-de0c07103e91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:06.147010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:06.147010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:06.147010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:06.147010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:06.147010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:06.147010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:06.147010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:06.147010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:06.147010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:06.147010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:06.147010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:06.147010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:06.147010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:06.147010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:06.148151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:06.148151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:06.148151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:06.148151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:06.148151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:06.148151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:06.148151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:06.148151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:06.148151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:06.148151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:06.148151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:06.148151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:06.148151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:06.148151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:06.148151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:06.148151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:06.148151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:06.150111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:06.150111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:06.150111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:06.150111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:06.150111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:06.150111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:06.150111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:06.150111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:06.152150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 304/599] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:06.157576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cfbf9-b09f-41ac-aa2c-de0c07103e91 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:06.181734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635ef75-a079-46b3-8d3c-ce4427ad3d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:06.182562 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635ef75-a079-46b3-8d3c-ce4427ad3d0c None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:06.182872 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635ef75-a079-46b3-8d3c-ce4427ad3d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:06.183245 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635ef75-a079-46b3-8d3c-ce4427ad3d0c None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:06.184265 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635ef75-a079-46b3-8d3c-ce4427ad3d0c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:06.184651 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635ef75-a079-46b3-8d3c-ce4427ad3d0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:06.186469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635ef75-a079-46b3-8d3c-ce4427ad3d0c None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:06.207674 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 296/600] 10.222.0.100 () {56 vars in 1144 bytes} [Tue Aug 2 08:58:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:06.213743 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635ef75-a079-46b3-8d3c-ce4427ad3d0c None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:06.271470 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cfbf9-b09f-41ac-aa2c-de0c07103e91 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:06.297136 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:06.338989 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d58794-c052-4044-a870-204dceab23e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:06.339621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d58794-c052-4044-a870-204dceab23e8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:06.339909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d58794-c052-4044-a870-204dceab23e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:06.340175 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d58794-c052-4044-a870-204dceab23e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:06.340922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d58794-c052-4044-a870-204dceab23e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:06.341252 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d58794-c052-4044-a870-204dceab23e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:06.347086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d58794-c052-4044-a870-204dceab23e8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:06.371880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 305/601] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:06.389210 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635ef75-a079-46b3-8d3c-ce4427ad3d0c None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:06.395364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d58794-c052-4044-a870-204dceab23e8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:06.486762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337523b-1bf1-41e9-be8d-d3ca7065f526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:06.487469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337523b-1bf1-41e9-be8d-d3ca7065f526 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:06.487689 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337523b-1bf1-41e9-be8d-d3ca7065f526 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:06.487968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337523b-1bf1-41e9-be8d-d3ca7065f526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:06.519195 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337523b-1bf1-41e9-be8d-d3ca7065f526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:06.520659 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337523b-1bf1-41e9-be8d-d3ca7065f526 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:06.535336 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d58794-c052-4044-a870-204dceab23e8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:06.550778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-f337523b-1bf1-41e9-be8d-d3ca7065f526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:06.550778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:06.550778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:06.550778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:06.550778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:06.550778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:06.550778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:06.550778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:06.550778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:06.550778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:06.550778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:06.550778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:06.550778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:06.550778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:06.550778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:06.551726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:06.551726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:06.551726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:06.551726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:06.551726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:06.551726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:06.551726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:06.551726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:06.551726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:06.551726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:06.551726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:06.551726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:06.551726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:06.551726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:06.551726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:06.551726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:06.551726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:06.552621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:06.552621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:06.552621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:06.552621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:06.557237 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 297/602] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:06.562106 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337523b-1bf1-41e9-be8d-d3ca7065f526 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:06.578506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:06.580591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0073b-c99c-439b-a73c-6ee03cfd9846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:06.581373 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0073b-c99c-439b-a73c-6ee03cfd9846 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:06.585436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0073b-c99c-439b-a73c-6ee03cfd9846 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:06.585436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0073b-c99c-439b-a73c-6ee03cfd9846 None admin] PATH_INFO: `/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/d959ddd087724547a11c3d4c796d8ab2/roles/6e6ac8e8e7044b209d1a8b5bb6e4b1d8` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:06.609340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c0073b-c99c-439b-a73c-6ee03cfd9846 None admin] RBAC: Authorizing `identity:create_grant(project_id=61ca0176324e445b94e42a5214d2c89b, user_id=d959ddd087724547a11c3d4c796d8ab2, role_id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:06.610785 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c0073b-c99c-439b-a73c-6ee03cfd9846 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:06.647663 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 306/603] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:58:06 2022] PUT /identity/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/d959ddd087724547a11c3d4c796d8ab2/roles/6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 08:58:06.671266 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337523b-1bf1-41e9-be8d-d3ca7065f526 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:06.710610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933fb6a-b953-4b83-bd26-bccda4c0a619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:06.711236 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933fb6a-b953-4b83-bd26-bccda4c0a619 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:06.712100 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933fb6a-b953-4b83-bd26-bccda4c0a619 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:06.712560 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933fb6a-b953-4b83-bd26-bccda4c0a619 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:06.713495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933fb6a-b953-4b83-bd26-bccda4c0a619 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:06.713808 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933fb6a-b953-4b83-bd26-bccda4c0a619 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:06.715234 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933fb6a-b953-4b83-bd26-bccda4c0a619 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:06.733291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 298/604] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:06.738296 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0073b-c99c-439b-a73c-6ee03cfd9846 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:06.759299 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a689c-db56-477e-8cac-d79ebd6cbc34 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:06.759667 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a689c-db56-477e-8cac-d79ebd6cbc34 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:06.759868 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a689c-db56-477e-8cac-d79ebd6cbc34 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:06.761506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 299/605] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:06 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:06.766823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9fbf4-02d4-40cd-841f-8468b3a338f9 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:06.767108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9fbf4-02d4-40cd-841f-8468b3a338f9 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:06.767386 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9fbf4-02d4-40cd-841f-8468b3a338f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:06.779898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:06.786927 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-5ba9fbf4-02d4-40cd-841f-8468b3a338f9 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:06.788058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-5ba9fbf4-02d4-40cd-841f-8468b3a338f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:06.861923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:06.875040 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 300/606] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:06 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:58:06.892358 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0073b-c99c-439b-a73c-6ee03cfd9846 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:06.897830 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9fbf4-02d4-40cd-841f-8468b3a338f9 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:06.948661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc61c18-c9a1-48bb-b131-604ce0928fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:06.949737 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc61c18-c9a1-48bb-b131-604ce0928fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:06.950338 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc61c18-c9a1-48bb-b131-604ce0928fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:06.951578 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc61c18-c9a1-48bb-b131-604ce0928fd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:06.960072 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc61c18-c9a1-48bb-b131-604ce0928fd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:06.962679 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc61c18-c9a1-48bb-b131-604ce0928fd6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:06.978714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ffc61c18-c9a1-48bb-b131-604ce0928fd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:06.978714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:06.978714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:06.978714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:06.978714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:06.978714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:06.978714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:06.978714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:06.978714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:06.978714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:06.978714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:06.978714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:06.978714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:06.978714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:06.978714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:06.979627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:06.979627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:06.979627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:06.979627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:06.979627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:06.979627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:06.979627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:06.979627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:06.979627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:06.979627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:06.979627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:06.979627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:06.979627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:06.979627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:06.979627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:06.979627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:06.979627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:06.980538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:06.980538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:06.980538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:06.980538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:06.980538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:06.980538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:06.980538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:06.980538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:06.982926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 307/607] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:06.988653 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc61c18-c9a1-48bb-b131-604ce0928fd6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:07.044937 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9fbf4-02d4-40cd-841f-8468b3a338f9 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:07.118608 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:07.119612 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250ecae-7ca5-4bac-97f6-e7c3fcbf02df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:07.120261 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250ecae-7ca5-4bac-97f6-e7c3fcbf02df None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:07.120542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250ecae-7ca5-4bac-97f6-e7c3fcbf02df None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:07.120816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250ecae-7ca5-4bac-97f6-e7c3fcbf02df None admin] PATH_INFO: `/v3/roles/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:07.144061 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b250ecae-7ca5-4bac-97f6-e7c3fcbf02df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:07.153124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc61c18-c9a1-48bb-b131-604ce0928fd6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:07.154934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-b250ecae-7ca5-4bac-97f6-e7c3fcbf02df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:07.154934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:07.154934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:07.154934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:07.154934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:07.154934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:07.154934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:07.154934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:07.154934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:07.154934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:07.154934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:07.154934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:07.154934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:07.154934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:07.154934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:07.154934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:07.155858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:07.155858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:07.155858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:07.155858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:07.155858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:07.155858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:07.155858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:07.155858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:07.155858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:07.155858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:07.155858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:07.155858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:07.155858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:07.155858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:07.155858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:07.155858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:07.156715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:07.156715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:07.156715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:07.156715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:07.156715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:07.156715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:07.156715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:07.156715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:07.156715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:07.156715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:07.156715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:07.156715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:07.158158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 301/608] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:58:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:07.163623 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250ecae-7ca5-4bac-97f6-e7c3fcbf02df None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:07.220090 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60a517-a985-4fd8-96a4-b9c53e0fe3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:07.220763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60a517-a985-4fd8-96a4-b9c53e0fe3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:07.221052 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60a517-a985-4fd8-96a4-b9c53e0fe3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:07.221358 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60a517-a985-4fd8-96a4-b9c53e0fe3b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:07.222115 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60a517-a985-4fd8-96a4-b9c53e0fe3b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:07.222424 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60a517-a985-4fd8-96a4-b9c53e0fe3b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:07.223868 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60a517-a985-4fd8-96a4-b9c53e0fe3b1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:07.245250 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 308/609] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:07.246832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c2f47-4034-46fb-973c-5b8c5ceb8f49 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:07.248134 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c2f47-4034-46fb-973c-5b8c5ceb8f49 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:07.249529 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c2f47-4034-46fb-973c-5b8c5ceb8f49 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:07.252209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 309/610] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:07 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:07.253259 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c2f47-4034-46fb-973c-5b8c5ceb8f49 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:07.285518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:07.315677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250ecae-7ca5-4bac-97f6-e7c3fcbf02df None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:07.359449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c3c75-720f-4877-be61-087f90dce31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:07.360556 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c3c75-720f-4877-be61-087f90dce31d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:07.360920 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c3c75-720f-4877-be61-087f90dce31d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:07.361301 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c3c75-720f-4877-be61-087f90dce31d None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:07.362463 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c3c75-720f-4877-be61-087f90dce31d None admin] RBAC: Adding query filter params (name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:07.363268 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c3c75-720f-4877-be61-087f90dce31d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:07.364983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c3c75-720f-4877-be61-087f90dce31d None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:07.367087 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c2f47-4034-46fb-973c-5b8c5ceb8f49 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:07.382617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 302/611] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 08:58:07 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:07.384224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb12c2-31a8-44d1-8576-5933a74ae78e None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:07.384513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb12c2-31a8-44d1-8576-5933a74ae78e None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:07.384787 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb12c2-31a8-44d1-8576-5933a74ae78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:07.402488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-3beb12c2-31a8-44d1-8576-5933a74ae78e None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:07.403670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3beb12c2-31a8-44d1-8576-5933a74ae78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:07.415150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c297a1-3ee2-47ad-ad98-508c81a01037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:07.417073 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c297a1-3ee2-47ad-ad98-508c81a01037 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:07.417670 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c297a1-3ee2-47ad-ad98-508c81a01037 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:07.418150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c297a1-3ee2-47ad-ad98-508c81a01037 None admin] PATH_INFO: `/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/f4afef33deb247509765a8c88a78ac39/roles/6e6ac8e8e7044b209d1a8b5bb6e4b1d8` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:07.434777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:07.439442 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c297a1-3ee2-47ad-ad98-508c81a01037 None admin] RBAC: Authorizing `identity:create_grant(project_id=61ca0176324e445b94e42a5214d2c89b, user_id=f4afef33deb247509765a8c88a78ac39, role_id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:07.442118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c297a1-3ee2-47ad-ad98-508c81a01037 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:07.465381 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 310/612] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:58:07 2022] PUT /identity/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/f4afef33deb247509765a8c88a78ac39/roles/6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:58:07.466785 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c297a1-3ee2-47ad-ad98-508c81a01037 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:07.504150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 303/613] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:58:07.545586 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb12c2-31a8-44d1-8576-5933a74ae78e None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:07.635157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:07.658983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c297a1-3ee2-47ad-ad98-508c81a01037 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:07.684512 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb12c2-31a8-44d1-8576-5933a74ae78e None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:07.722551 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f83a9-4a8f-4209-9496-77d57ec943c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:07.723588 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f83a9-4a8f-4209-9496-77d57ec943c2 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:07.724077 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f83a9-4a8f-4209-9496-77d57ec943c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:07.724588 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f83a9-4a8f-4209-9496-77d57ec943c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:07.728020 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2cdb8-9be4-43b1-a05c-27e1edfd426e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:07.728870 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2cdb8-9be4-43b1-a05c-27e1edfd426e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:07.729171 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2cdb8-9be4-43b1-a05c-27e1edfd426e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:07.729449 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2cdb8-9be4-43b1-a05c-27e1edfd426e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:07.738368 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f83a9-4a8f-4209-9496-77d57ec943c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:07.744093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-665f83a9-4a8f-4209-9496-77d57ec943c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:07.744093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:07.744093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:07.744093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:07.744093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:07.744093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:07.744093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:07.744093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:07.744093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:07.744093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:07.744093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:07.744093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:07.744093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:07.744093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:07.744093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:07.744093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:07.745041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:07.745041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:07.745041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:07.745041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:07.745041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:07.745041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:07.745041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:07.745041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:07.745041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:07.745041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:07.745041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:07.745041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:07.745041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:07.745041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:07.745041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:07.745041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:07.746002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:07.746002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:07.746002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:07.746002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:07.746002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:07.746002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:07.746002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:07.746002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:07.746002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:07.746002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:07.746002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:07.746002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:07.753141 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2cdb8-9be4-43b1-a05c-27e1edfd426e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:07.753141 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2cdb8-9be4-43b1-a05c-27e1edfd426e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:07.754031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 304/614] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:58:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:07.756128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f83a9-4a8f-4209-9496-77d57ec943c2 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:07.772515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-e6d2cdb8-9be4-43b1-a05c-27e1edfd426e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:07.772515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:07.772515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:07.772515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:07.772515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:07.772515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:07.772515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:07.772515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:07.772515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:07.772515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:07.772515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:07.772515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:07.772515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:07.772515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:07.772515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:07.774389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:07.774389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:07.774389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:07.774389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:07.774389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:07.774389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:07.774389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:07.774389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:07.774389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:07.774389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:07.774389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:07.774389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:07.774389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:07.774389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:07.774389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:07.774389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:07.774389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:07.775766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:07.775766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:07.775766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:07.775766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:07.775766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 311/615] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:07.780330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2cdb8-9be4-43b1-a05c-27e1edfd426e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:07.836076 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:07.866868 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f83a9-4a8f-4209-9496-77d57ec943c2 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:07.881738 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:07.903750 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2cdb8-9be4-43b1-a05c-27e1edfd426e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:07.921711 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d4196-3a36-4af3-9710-bf069dd4339b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:07.922456 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d4196-3a36-4af3-9710-bf069dd4339b None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:07.922842 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d4196-3a36-4af3-9710-bf069dd4339b None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:07.923222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d4196-3a36-4af3-9710-bf069dd4339b None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:07.924054 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d4196-3a36-4af3-9710-bf069dd4339b None admin] RBAC: Adding query filter params (name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:07.925457 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d4196-3a36-4af3-9710-bf069dd4339b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:07.927600 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d4196-3a36-4af3-9710-bf069dd4339b None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:07.943345 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 305/616] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 08:58:07 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:07.950508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d4196-3a36-4af3-9710-bf069dd4339b None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:07.984354 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02fa0e7-b31d-495f-aa6d-366a03fa3426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:07.984997 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fa0e7-b31d-495f-aa6d-366a03fa3426 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:07.985306 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fa0e7-b31d-495f-aa6d-366a03fa3426 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:07.985586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fa0e7-b31d-495f-aa6d-366a03fa3426 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:07.986291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02fa0e7-b31d-495f-aa6d-366a03fa3426 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:07.986599 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02fa0e7-b31d-495f-aa6d-366a03fa3426 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:07.988132 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02fa0e7-b31d-495f-aa6d-366a03fa3426 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:08.003868 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 312/617] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:08.010404 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02fa0e7-b31d-495f-aa6d-366a03fa3426 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:08.097142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d4196-3a36-4af3-9710-bf069dd4339b None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:08.156495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61eb92-07c3-47c4-9fb2-3c2324b218ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:08.160179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61eb92-07c3-47c4-9fb2-3c2324b218ca None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:08.160580 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61eb92-07c3-47c4-9fb2-3c2324b218ca None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:08.160986 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61eb92-07c3-47c4-9fb2-3c2324b218ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:08.180812 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a61eb92-07c3-47c4-9fb2-3c2324b218ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:08.183397 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a61eb92-07c3-47c4-9fb2-3c2324b218ca None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:08.212508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-4a61eb92-07c3-47c4-9fb2-3c2324b218ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:08.212508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:08.212508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:08.212508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:08.212508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:08.212508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:08.212508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:08.212508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:08.212508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:08.212508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:08.212508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:08.212508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:08.212508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:08.212508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:08.212508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:08.213633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:08.213633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:08.213633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:08.213633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:08.213633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:08.213633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:08.213633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:08.213633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:08.213633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:08.213633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:08.213633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:08.213633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:08.213633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:08.213633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:08.213633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:08.213633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:08.213633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:08.214714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:08.214714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:08.214714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:08.214714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:08.224374 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 306/618] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:08.226481 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02fa0e7-b31d-495f-aa6d-366a03fa3426 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:08.229186 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:08.229507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61eb92-07c3-47c4-9fb2-3c2324b218ca None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:08.265135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:08.285270 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19c91a-208d-46d7-8c2c-4a57371aa682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:08.286168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19c91a-208d-46d7-8c2c-4a57371aa682 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:08.286454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19c91a-208d-46d7-8c2c-4a57371aa682 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:08.286727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19c91a-208d-46d7-8c2c-4a57371aa682 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:08.296549 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da19c91a-208d-46d7-8c2c-4a57371aa682 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:08.298287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da19c91a-208d-46d7-8c2c-4a57371aa682 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:08.316120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-da19c91a-208d-46d7-8c2c-4a57371aa682 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:58:08.316120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:08.316120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:08.316120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:08.316120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:08.316120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:08.316120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:08.316120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:08.316120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:08.316120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:08.316120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:08.316120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:08.316120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:08.316120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:08.316120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:08.316120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:08.317346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:08.317346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:08.317346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:08.317346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:08.317346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:08.317346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:08.317346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:08.317346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:08.317346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:08.317346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:08.317346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:08.317346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:08.317346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:08.317346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:08.317346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:08.317346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:08.317346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:08.317346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:08.318436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:08.318436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:08.318436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:08.318436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:08.318436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:08.318436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:08.318436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:08.318436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:08.318436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:58:08.318436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:08.318436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 313/619] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:58:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:08.319044 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7210c-4b98-4ae6-afec-dccdc5529169 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:08.319044 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7210c-4b98-4ae6-afec-dccdc5529169 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:08.319388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7210c-4b98-4ae6-afec-dccdc5529169 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:08.325227 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 314/620] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:08 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:08.326521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7210c-4b98-4ae6-afec-dccdc5529169 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:08.354782 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61eb92-07c3-47c4-9fb2-3c2324b218ca None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:08.396874 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668207d-e8d0-44bb-a710-754190d2eb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:08.397995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668207d-e8d0-44bb-a710-754190d2eb29 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:08.398330 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668207d-e8d0-44bb-a710-754190d2eb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:08.399480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668207d-e8d0-44bb-a710-754190d2eb29 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:08.400177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668207d-e8d0-44bb-a710-754190d2eb29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:08.400634 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668207d-e8d0-44bb-a710-754190d2eb29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:08.402729 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668207d-e8d0-44bb-a710-754190d2eb29 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:08.425037 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 307/621] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:08.426356 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e0f7c-75fd-4568-a79d-1174a58e71b2 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:08.427006 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e0f7c-75fd-4568-a79d-1174a58e71b2 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:08.427299 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e0f7c-75fd-4568-a79d-1174a58e71b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:08.450727 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-c36e0f7c-75fd-4568-a79d-1174a58e71b2 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:08.451965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c36e0f7c-75fd-4568-a79d-1174a58e71b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:08.452524 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7210c-4b98-4ae6-afec-dccdc5529169 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:08.493357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceebe0a-1e9e-43f9-a2eb-d56daf7236c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:08.494296 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceebe0a-1e9e-43f9-a2eb-d56daf7236c1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:08.494641 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceebe0a-1e9e-43f9-a2eb-d56daf7236c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:08.495009 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceebe0a-1e9e-43f9-a2eb-d56daf7236c1 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:08.496085 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceebe0a-1e9e-43f9-a2eb-d56daf7236c1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:08.496396 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceebe0a-1e9e-43f9-a2eb-d56daf7236c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:08.498002 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceebe0a-1e9e-43f9-a2eb-d56daf7236c1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:08.521615 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 315/622] 10.222.0.100 () {56 vars in 1140 bytes} [Tue Aug 2 08:58:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:08.522529 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceebe0a-1e9e-43f9-a2eb-d56daf7236c1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:08.544076 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 308/623] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:58:08.544953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e0f7c-75fd-4568-a79d-1174a58e71b2 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:08.568195 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:08.580195 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:08.665482 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceebe0a-1e9e-43f9-a2eb-d56daf7236c1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:08.679664 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e0f7c-75fd-4568-a79d-1174a58e71b2 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:08.709801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6a045-85cd-46e4-abd1-d84774f4c338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:08.710455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6a045-85cd-46e4-abd1-d84774f4c338 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:08.710737 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6a045-85cd-46e4-abd1-d84774f4c338 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:08.711192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6a045-85cd-46e4-abd1-d84774f4c338 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:08.716946 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e858a-0291-4930-91a9-b15b42d9b62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:08.717710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e858a-0291-4930-91a9-b15b42d9b62b None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:08.718086 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e858a-0291-4930-91a9-b15b42d9b62b None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:08.718451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e858a-0291-4930-91a9-b15b42d9b62b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:08.719897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6a045-85cd-46e4-abd1-d84774f4c338 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:08.721857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6a045-85cd-46e4-abd1-d84774f4c338 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:08.729266 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e858a-0291-4930-91a9-b15b42d9b62b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:08.730828 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e858a-0291-4930-91a9-b15b42d9b62b None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:08.735521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-fff6a045-85cd-46e4-abd1-d84774f4c338 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:58:08.735521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:08.735521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:08.735521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:08.735521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:08.735521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:08.735521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:08.735521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:08.735521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:08.735521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:08.735521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:08.735521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:08.735521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:08.735521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:08.735521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:08.735521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:08.736640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:08.736640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:08.736640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:08.736640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:08.736640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:08.736640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:08.736640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:08.736640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:08.736640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:08.736640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:08.736640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:08.736640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:08.736640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:08.736640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:08.736640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:08.736640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:08.736640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:08.736640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:08.737775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:08.737775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:08.737775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:08.737775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:08.737775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:08.737775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:08.737775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:08.737775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:08.737775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:58:08.737775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:08.737775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 316/624] 10.222.0.100 () {56 vars in 1098 bytes} [Tue Aug 2 08:58:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:08.737775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6a045-85cd-46e4-abd1-d84774f4c338 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:08.749968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-886e858a-0291-4930-91a9-b15b42d9b62b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:08.749968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:08.749968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:08.749968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:08.749968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:08.749968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:08.749968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:08.749968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:08.749968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:08.749968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:08.749968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:08.749968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:08.749968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:08.749968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:08.749968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:08.751227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:08.751227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:08.751227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:08.751227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:08.751227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:08.751227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:08.751227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:08.751227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:08.751227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:08.751227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:08.751227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:08.751227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:08.751227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:08.751227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:08.751227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:08.751227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:08.751227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:08.752309 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:08.752309 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:08.752309 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:08.752309 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:08.758675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 309/625] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:08.759650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e858a-0291-4930-91a9-b15b42d9b62b None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:08.838492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:08.853544 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:08.860643 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6a045-85cd-46e4-abd1-d84774f4c338 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:08.870779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e858a-0291-4930-91a9-b15b42d9b62b None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:08.903701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d55d0f-f1fa-41f6-be55-a8cf8ee9ce57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:08.904429 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d55d0f-f1fa-41f6-be55-a8cf8ee9ce57 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:08.904717 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d55d0f-f1fa-41f6-be55-a8cf8ee9ce57 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:08.904995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d55d0f-f1fa-41f6-be55-a8cf8ee9ce57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:08.908939 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed888df3-2338-4a29-8862-e1bed7ac0964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:08.909986 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed888df3-2338-4a29-8862-e1bed7ac0964 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:08.910479 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed888df3-2338-4a29-8862-e1bed7ac0964 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:08.911080 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed888df3-2338-4a29-8862-e1bed7ac0964 None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:08.912075 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed888df3-2338-4a29-8862-e1bed7ac0964 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:08.912598 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed888df3-2338-4a29-8862-e1bed7ac0964 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:08.914917 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed888df3-2338-4a29-8862-e1bed7ac0964 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:08.920598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d55d0f-f1fa-41f6-be55-a8cf8ee9ce57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:08.930468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-74d55d0f-f1fa-41f6-be55-a8cf8ee9ce57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:08.930468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:08.930468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:08.930468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:08.930468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:08.930468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:08.930468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:08.930468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:08.930468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:08.930468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:08.930468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:08.930468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:08.930468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:08.930468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:08.930468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:08.930468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:08.931440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:08.931440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:08.931440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:08.931440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:08.931440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:08.931440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:08.931440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:08.931440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:08.931440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:08.931440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:08.931440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:08.931440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:08.931440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:08.931440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:08.931440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:08.931440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:08.932282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:08.932282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:08.932282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:08.932282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:08.932282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:08.932282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:08.932282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:08.932282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:08.932282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:08.932282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:08.932282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:08.932282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:08.936541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 317/626] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:58:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:08.937445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d55d0f-f1fa-41f6-be55-a8cf8ee9ce57 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:08.942698 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 310/627] 10.222.0.100 () {56 vars in 1138 bytes} [Tue Aug 2 08:58:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:08.944002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed888df3-2338-4a29-8862-e1bed7ac0964 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:09.055865 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed888df3-2338-4a29-8862-e1bed7ac0964 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:09.062132 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d55d0f-f1fa-41f6-be55-a8cf8ee9ce57 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:09.082626 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:09.092769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439369d-cdd7-4c97-80a7-ef021f31205d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:09.093538 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439369d-cdd7-4c97-80a7-ef021f31205d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:09.093817 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439369d-cdd7-4c97-80a7-ef021f31205d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:09.094095 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439369d-cdd7-4c97-80a7-ef021f31205d None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:09.094800 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439369d-cdd7-4c97-80a7-ef021f31205d None admin] RBAC: Adding query filter params (name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:09.095103 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439369d-cdd7-4c97-80a7-ef021f31205d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:09.096457 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439369d-cdd7-4c97-80a7-ef021f31205d None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:09.097937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba15016-5f39-42c9-818a-889c3cb77de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:09.098622 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba15016-5f39-42c9-818a-889c3cb77de5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:09.098968 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba15016-5f39-42c9-818a-889c3cb77de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:09.099310 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba15016-5f39-42c9-818a-889c3cb77de5 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:09.100064 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba15016-5f39-42c9-818a-889c3cb77de5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:09.100439 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba15016-5f39-42c9-818a-889c3cb77de5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:09.102400 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba15016-5f39-42c9-818a-889c3cb77de5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:09.105443 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 311/628] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 08:58:08 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:09.106035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439369d-cdd7-4c97-80a7-ef021f31205d None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:09.120513 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 318/629] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:09.121691 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b91a9-b669-438b-bdc6-14586e7e9014 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:09.122031 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b91a9-b669-438b-bdc6-14586e7e9014 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:09.122365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b91a9-b669-438b-bdc6-14586e7e9014 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:09.125236 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 319/630] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:09.125236 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b91a9-b669-438b-bdc6-14586e7e9014 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:09.128503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:09.219752 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439369d-cdd7-4c97-80a7-ef021f31205d None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:09.229798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b91a9-b669-438b-bdc6-14586e7e9014 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:09.254844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac884cdc-d6e0-4f96-995c-0814bea00e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:09.255455 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac884cdc-d6e0-4f96-995c-0814bea00e20 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:09.255739 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac884cdc-d6e0-4f96-995c-0814bea00e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:09.256011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac884cdc-d6e0-4f96-995c-0814bea00e20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:09.264238 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54841b-2eb5-439f-aef0-785984a8cf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:09.264884 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54841b-2eb5-439f-aef0-785984a8cf3c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:09.265184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54841b-2eb5-439f-aef0-785984a8cf3c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:09.265457 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54841b-2eb5-439f-aef0-785984a8cf3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:09.276719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54841b-2eb5-439f-aef0-785984a8cf3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:09.278195 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54841b-2eb5-439f-aef0-785984a8cf3c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:09.293057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-7e54841b-2eb5-439f-aef0-785984a8cf3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:09.293057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:09.293057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:09.293057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:09.293057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:09.293057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:09.293057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:09.293057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:09.293057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:09.293057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:09.293057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:09.293057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:09.293057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:09.293057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:09.293057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:09.293998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:09.293998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:09.293998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:09.293998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:09.293998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:09.293998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:09.293998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:09.293998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:09.293998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:09.293998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:09.293998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:09.293998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:09.293998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:09.293998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:09.293998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:09.293998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:09.293998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:09.295359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:09.295359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:09.295359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:09.295359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:09.295359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 320/631] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:09.295359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54841b-2eb5-439f-aef0-785984a8cf3c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:09.318052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:09.334869 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac884cdc-d6e0-4f96-995c-0814bea00e20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:09.336875 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac884cdc-d6e0-4f96-995c-0814bea00e20 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:09.357283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:09.407753 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54841b-2eb5-439f-aef0-785984a8cf3c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:09.443740 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e5d6a-cfb9-4345-8d1f-003ed503dd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:09.448980 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 312/632] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:58:09 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:58:09.449144 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e5d6a-cfb9-4345-8d1f-003ed503dd50 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:09.449144 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e5d6a-cfb9-4345-8d1f-003ed503dd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:09.449144 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e5d6a-cfb9-4345-8d1f-003ed503dd50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:09.449394 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb26164-60f2-4af7-a5e2-7ce346287a4c None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:09.449394 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb26164-60f2-4af7-a5e2-7ce346287a4c None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:09.449394 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb26164-60f2-4af7-a5e2-7ce346287a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:09.456353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e5d6a-cfb9-4345-8d1f-003ed503dd50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:09.457815 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e5d6a-cfb9-4345-8d1f-003ed503dd50 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:09.465948 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-3fb26164-60f2-4af7-a5e2-7ce346287a4c None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:09.467171 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3fb26164-60f2-4af7-a5e2-7ce346287a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:09.469907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-4b6e5d6a-cfb9-4345-8d1f-003ed503dd50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:09.469907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:09.469907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:09.469907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:09.469907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:09.469907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:09.469907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:09.469907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:09.469907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:09.469907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:09.469907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:09.469907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:09.469907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:09.469907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:09.469907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:09.470841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:09.470841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:09.470841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:09.470841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:09.470841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:09.470841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:09.470841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:09.470841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:09.470841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:09.470841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:09.470841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:09.470841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:09.470841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:09.470841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:09.470841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:09.470841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:09.470841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:09.471743 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:09.471743 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:09.471743 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:09.471743 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:09.471743 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:09.471743 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:09.471743 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:09.471743 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:09.471743 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 321/633] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:09.471743 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e5d6a-cfb9-4345-8d1f-003ed503dd50 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:09.557161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 313/634] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:09 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:58:09.558125 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb26164-60f2-4af7-a5e2-7ce346287a4c None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:09.578790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:09.582100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e5d6a-cfb9-4345-8d1f-003ed503dd50 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:09.598236 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:09.619576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a302238-52b5-4735-8e7a-a70ca77460b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:09.620199 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a302238-52b5-4735-8e7a-a70ca77460b5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:09.620485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a302238-52b5-4735-8e7a-a70ca77460b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:09.620769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a302238-52b5-4735-8e7a-a70ca77460b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:09.621487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a302238-52b5-4735-8e7a-a70ca77460b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:09.621799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a302238-52b5-4735-8e7a-a70ca77460b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:09.623194 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a302238-52b5-4735-8e7a-a70ca77460b5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:09.636806 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 322/635] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:09.637670 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a302238-52b5-4735-8e7a-a70ca77460b5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:09.671549 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb26164-60f2-4af7-a5e2-7ce346287a4c None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:09.708265 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd051ca-4192-467b-bbfa-292f2b6ac39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:09.708905 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd051ca-4192-467b-bbfa-292f2b6ac39d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:09.709219 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd051ca-4192-467b-bbfa-292f2b6ac39d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:09.709494 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd051ca-4192-467b-bbfa-292f2b6ac39d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:09.721868 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd051ca-4192-467b-bbfa-292f2b6ac39d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:09.723248 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd051ca-4192-467b-bbfa-292f2b6ac39d None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:09.739307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-4bd051ca-4192-467b-bbfa-292f2b6ac39d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:09.739307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:09.739307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:09.739307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:09.739307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:09.739307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:09.739307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:09.739307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:09.739307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:09.739307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:09.739307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:09.739307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:09.739307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:09.739307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:09.739307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:09.740275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:09.740275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:09.740275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:09.740275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:09.740275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:09.740275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:09.740275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:09.740275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:09.740275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:09.740275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:09.740275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:09.740275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:09.740275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:09.740275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:09.740275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:09.740275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:09.740275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:09.741216 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:09.741216 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:09.741216 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:09.741216 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:09.741216 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 314/636] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:09.742074 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd051ca-4192-467b-bbfa-292f2b6ac39d None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:09.750163 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a302238-52b5-4735-8e7a-a70ca77460b5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:09.788824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca2f45-6b43-4186-9624-d7ff9dd02970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:09.790395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca2f45-6b43-4186-9624-d7ff9dd02970 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:09.791212 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca2f45-6b43-4186-9624-d7ff9dd02970 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:09.791670 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca2f45-6b43-4186-9624-d7ff9dd02970 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:09.792562 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca2f45-6b43-4186-9624-d7ff9dd02970 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:09.793090 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca2f45-6b43-4186-9624-d7ff9dd02970 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:09.794550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca2f45-6b43-4186-9624-d7ff9dd02970 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:09.813216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 323/637] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:09.813216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca2f45-6b43-4186-9624-d7ff9dd02970 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:09.818124 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:09.850974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:09.852213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd051ca-4192-467b-bbfa-292f2b6ac39d None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:09.888353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b2752-0c11-44bf-b847-c470b2b27b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:09.888992 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b2752-0c11-44bf-b847-c470b2b27b2a None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:09.889307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b2752-0c11-44bf-b847-c470b2b27b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:09.889593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b2752-0c11-44bf-b847-c470b2b27b2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:09.899645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b2752-0c11-44bf-b847-c470b2b27b2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:09.905058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-627b2752-0c11-44bf-b847-c470b2b27b2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:09.905058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:09.905058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:09.905058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:09.905058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:09.905058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:09.905058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:09.905058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:09.905058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:09.905058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:09.905058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:09.905058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:09.905058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:09.905058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:09.905058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:09.905058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:09.906060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:09.906060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:09.906060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:09.906060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:09.906060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:09.906060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:09.906060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:09.906060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:09.906060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:09.906060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:09.906060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:09.906060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:09.906060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:09.906060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:09.906060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:09.906060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:09.910069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:09.910069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:09.910069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:09.910069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:09.910069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:09.910069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:09.910069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:09.910069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:09.910069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:09.910069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:09.910069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:09.910069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:09.910069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 315/638] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:58:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:09.910069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b2752-0c11-44bf-b847-c470b2b27b2a None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:09.924445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca2f45-6b43-4186-9624-d7ff9dd02970 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:09.958050 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170ee52-ac16-4825-bb7b-b6ef0cabf2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:09.958694 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170ee52-ac16-4825-bb7b-b6ef0cabf2ac None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:09.958982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170ee52-ac16-4825-bb7b-b6ef0cabf2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:09.959288 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170ee52-ac16-4825-bb7b-b6ef0cabf2ac None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:09.966647 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170ee52-ac16-4825-bb7b-b6ef0cabf2ac None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:09.968082 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170ee52-ac16-4825-bb7b-b6ef0cabf2ac None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:09.979818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-b170ee52-ac16-4825-bb7b-b6ef0cabf2ac None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 08:58:09.979818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:09.979818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:09.979818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:09.979818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:09.979818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:09.979818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:09.979818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:09.979818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:09.979818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:09.979818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:09.979818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:09.979818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:09.979818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:09.979818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:09.979818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:09.980836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:09.980836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:09.980836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:09.980836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:09.980836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:09.980836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:09.980836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:09.980836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:09.980836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:09.980836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:09.980836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:09.980836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:09.980836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:09.980836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:09.980836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:09.980836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:09.980836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:09.980836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:09.981854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:09.981854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:09.981854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:09.981854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:09.981854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:09.981854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:09.981854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:09.981854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:09.981854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 08:58:09.981854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:09.981854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 324/639] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:58:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:09.982460 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170ee52-ac16-4825-bb7b-b6ef0cabf2ac None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:10.016772 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b2752-0c11-44bf-b847-c470b2b27b2a None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:10.046220 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:10.052434 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0497b4-6337-40b0-a34f-15236ab1fac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:10.053033 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0497b4-6337-40b0-a34f-15236ab1fac5 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:10.053390 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0497b4-6337-40b0-a34f-15236ab1fac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:10.053674 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0497b4-6337-40b0-a34f-15236ab1fac5 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:10.054361 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0497b4-6337-40b0-a34f-15236ab1fac5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:10.054666 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0497b4-6337-40b0-a34f-15236ab1fac5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:10.056236 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0497b4-6337-40b0-a34f-15236ab1fac5 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:10.067902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 316/640] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:10.069065 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395cf3c-34ed-4368-8b68-ee117df0877e None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:10.069380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395cf3c-34ed-4368-8b68-ee117df0877e None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:10.069662 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395cf3c-34ed-4368-8b68-ee117df0877e None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:10.072743 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 317/641] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:10.072743 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395cf3c-34ed-4368-8b68-ee117df0877e None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:10.083304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:10.091456 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170ee52-ac16-4825-bb7b-b6ef0cabf2ac None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:10.124803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfd439-b8bf-405f-8ed0-33e32acf564a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:10.125516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfd439-b8bf-405f-8ed0-33e32acf564a None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:10.126063 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfd439-b8bf-405f-8ed0-33e32acf564a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:10.126343 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfd439-b8bf-405f-8ed0-33e32acf564a None admin] PATH_INFO: `/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/50cb532b58ed4951ba960a0a7d579b49/roles/6e6ac8e8e7044b209d1a8b5bb6e4b1d8` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:10.140731 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebfd439-b8bf-405f-8ed0-33e32acf564a None admin] RBAC: Authorizing `identity:create_grant(project_id=61ca0176324e445b94e42a5214d2c89b, user_id=50cb532b58ed4951ba960a0a7d579b49, role_id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:10.142202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebfd439-b8bf-405f-8ed0-33e32acf564a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:10.155418 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 325/642] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:58:09 2022] PUT /identity/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/50cb532b58ed4951ba960a0a7d579b49/roles/6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:58:10.156211 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfd439-b8bf-405f-8ed0-33e32acf564a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:10.181031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395cf3c-34ed-4368-8b68-ee117df0877e None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:10.221599 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f09db6-fb18-4b77-bf2c-3d9eff213e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:10.222240 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f09db6-fb18-4b77-bf2c-3d9eff213e15 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:10.222518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f09db6-fb18-4b77-bf2c-3d9eff213e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:10.222789 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f09db6-fb18-4b77-bf2c-3d9eff213e15 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:10.223520 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f09db6-fb18-4b77-bf2c-3d9eff213e15 None admin] RBAC: Adding query filter params (name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:10.223831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f09db6-fb18-4b77-bf2c-3d9eff213e15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:10.225395 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f09db6-fb18-4b77-bf2c-3d9eff213e15 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:10.236390 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 318/643] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 08:58:10 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:10.237251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f09db6-fb18-4b77-bf2c-3d9eff213e15 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:10.280826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfd439-b8bf-405f-8ed0-33e32acf564a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:10.300331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:10.318716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9de14-777a-47b7-b366-f92681241cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:10.319352 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9de14-777a-47b7-b366-f92681241cda None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:10.319678 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9de14-777a-47b7-b366-f92681241cda None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:10.319908 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9de14-777a-47b7-b366-f92681241cda None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:10.321495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9de14-777a-47b7-b366-f92681241cda None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:10.321626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9de14-777a-47b7-b366-f92681241cda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:10.323052 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9de14-777a-47b7-b366-f92681241cda None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:10.343485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 326/644] 10.222.0.100 () {56 vars in 1142 bytes} [Tue Aug 2 08:58:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:10.345584 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90d048-69a6-4656-a251-e15997285979 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:10.346244 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90d048-69a6-4656-a251-e15997285979 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:10.346539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90d048-69a6-4656-a251-e15997285979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:10.356755 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f09db6-fb18-4b77-bf2c-3d9eff213e15 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:10.359674 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:10.366014 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ee90d048-69a6-4656-a251-e15997285979 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:10.368504 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ee90d048-69a6-4656-a251-e15997285979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:10.396000 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6d5be-1b4c-4088-a00b-10c35a95c13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:10.396642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6d5be-1b4c-4088-a00b-10c35a95c13f None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:10.396927 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6d5be-1b4c-4088-a00b-10c35a95c13f None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:10.397232 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6d5be-1b4c-4088-a00b-10c35a95c13f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:10.404191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6d5be-1b4c-4088-a00b-10c35a95c13f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:10.405688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6d5be-1b4c-4088-a00b-10c35a95c13f None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:10.418270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-87e6d5be-1b4c-4088-a00b-10c35a95c13f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:10.418270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:10.418270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:10.418270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:10.418270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:10.418270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:10.418270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:10.418270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:10.418270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:10.418270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:10.418270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:10.418270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:10.418270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:10.418270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:10.418270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:10.419179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:10.419179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:10.419179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:10.419179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:10.419179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:10.419179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:10.419179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:10.419179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:10.419179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:10.419179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:10.419179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:10.419179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:10.419179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:10.419179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:10.419179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:10.419179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:10.419179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:10.420228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:10.420228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:10.420228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:10.420228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:10.420228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:10.420228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:10.420228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:10.420228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:10.420228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 319/645] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:10.420228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6d5be-1b4c-4088-a00b-10c35a95c13f None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:10.457137 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 327/646] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:10 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:58:10.457137 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90d048-69a6-4656-a251-e15997285979 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:10.541247 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6d5be-1b4c-4088-a00b-10c35a95c13f None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:10.555456 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:10.569036 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90d048-69a6-4656-a251-e15997285979 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:10.580580 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e1cd2-8813-4f59-9d53-840eedd8cbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:10.581929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e1cd2-8813-4f59-9d53-840eedd8cbb9 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:10.582290 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e1cd2-8813-4f59-9d53-840eedd8cbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:10.582643 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e1cd2-8813-4f59-9d53-840eedd8cbb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:10.594090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e1cd2-8813-4f59-9d53-840eedd8cbb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:10.595635 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e1cd2-8813-4f59-9d53-840eedd8cbb9 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:10.600958 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:10.608174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e9919-9db9-4d1f-9893-fc3664813702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:10.608806 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e9919-9db9-4d1f-9893-fc3664813702 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:10.609091 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e9919-9db9-4d1f-9893-fc3664813702 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:10.609399 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e9919-9db9-4d1f-9893-fc3664813702 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:10.611425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-839e1cd2-8813-4f59-9d53-840eedd8cbb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:10.611425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:10.611425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:10.611425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:10.611425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:10.611425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:10.611425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:10.611425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:10.611425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:10.611425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:10.611425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:10.611425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:10.611425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:10.611425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:10.611425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:10.612274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:10.612274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:10.612274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:10.612274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:10.612274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:10.612274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:10.612274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:10.612274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:10.612274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:10.612274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:10.612274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:10.612274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:10.612274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:10.612274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:10.612274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:10.612274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:10.612274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:10.613223 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:10.613223 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:10.613223 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:10.613223 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:10.618089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 320/647] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:10.618848 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e1cd2-8813-4f59-9d53-840eedd8cbb9 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:10.688248 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e9919-9db9-4d1f-9893-fc3664813702 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:10.690082 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e9919-9db9-4d1f-9893-fc3664813702 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:10.725944 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e1cd2-8813-4f59-9d53-840eedd8cbb9 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:10.763034 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bc70e-7447-45ad-a0b6-70840fa46681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:10.763652 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bc70e-7447-45ad-a0b6-70840fa46681 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:10.763935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bc70e-7447-45ad-a0b6-70840fa46681 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:10.764212 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bc70e-7447-45ad-a0b6-70840fa46681 None admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:10.764955 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bc70e-7447-45ad-a0b6-70840fa46681 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:10.765315 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bc70e-7447-45ad-a0b6-70840fa46681 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:10.766738 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bc70e-7447-45ad-a0b6-70840fa46681 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:10.785692 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 321/648] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:10.785692 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bc70e-7447-45ad-a0b6-70840fa46681 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:10.813576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 328/649] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:58:10 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:58:10.813576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e9919-9db9-4d1f-9893-fc3664813702 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:10.814816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:10.845974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:10.894886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bc70e-7447-45ad-a0b6-70840fa46681 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:10.931653 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e9919-9db9-4d1f-9893-fc3664813702 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:10.933385 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27632a9-9fc0-4a42-878c-b4ab983eecb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:10.934096 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27632a9-9fc0-4a42-878c-b4ab983eecb3 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:10.934441 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27632a9-9fc0-4a42-878c-b4ab983eecb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:10.934776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27632a9-9fc0-4a42-878c-b4ab983eecb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:10.945131 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27632a9-9fc0-4a42-878c-b4ab983eecb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:10.952670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-c27632a9-9fc0-4a42-878c-b4ab983eecb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:10.952670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:10.952670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:10.952670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:10.952670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:10.952670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:10.952670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:10.952670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:10.952670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:10.952670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:10.952670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:10.952670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:10.952670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:10.952670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:10.952670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:10.952670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:10.953862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:10.953862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:10.953862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:10.953862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:10.953862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:10.953862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:10.953862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:10.953862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:10.953862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:10.953862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:10.953862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:10.953862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:10.953862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:10.953862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:10.953862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:10.953862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:10.954806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:10.954806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:10.954806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:10.954806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:10.954806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:10.954806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:10.954806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:10.954806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:10.954806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:10.954806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:10.954806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:10.954806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:10.956144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 322/650] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:58:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:10.956999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27632a9-9fc0-4a42-878c-b4ab983eecb3 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:10.974627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cedbae-378a-4cf6-9f74-333b90c9ac7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:10.975246 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cedbae-378a-4cf6-9f74-333b90c9ac7c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:10.975523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cedbae-378a-4cf6-9f74-333b90c9ac7c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:10.975795 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cedbae-378a-4cf6-9f74-333b90c9ac7c None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:10.976485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cedbae-378a-4cf6-9f74-333b90c9ac7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:10.976943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cedbae-378a-4cf6-9f74-333b90c9ac7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:10.978349 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cedbae-378a-4cf6-9f74-333b90c9ac7c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:10.994578 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 329/651] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:10.994578 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cedbae-378a-4cf6-9f74-333b90c9ac7c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:11.055490 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:11.073871 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27632a9-9fc0-4a42-878c-b4ab983eecb3 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:11.086459 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:11.096234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cedbae-378a-4cf6-9f74-333b90c9ac7c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:11.109506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01430be-3923-4229-a8b1-406969f7cb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:11.110049 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01430be-3923-4229-a8b1-406969f7cb19 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:11.110332 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01430be-3923-4229-a8b1-406969f7cb19 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:11.110847 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01430be-3923-4229-a8b1-406969f7cb19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:11.111406 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01430be-3923-4229-a8b1-406969f7cb19 None admin] RBAC: Adding query filter params (role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=49a0c1a4a6cf43e28ec1b8488c681848) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:11.111716 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01430be-3923-4229-a8b1-406969f7cb19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:11.113184 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01430be-3923-4229-a8b1-406969f7cb19 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:11.129159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 323/652] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:58:10 2022] GET /identity/v3/role_assignments?user.id=49a0c1a4a6cf43e28ec1b8488c681848&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 610 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:11.129159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01430be-3923-4229-a8b1-406969f7cb19 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:11.143571 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3371ed-c362-4308-ba17-b1f4449b8294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:11.144336 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3371ed-c362-4308-ba17-b1f4449b8294 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:11.144693 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3371ed-c362-4308-ba17-b1f4449b8294 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:11.145066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3371ed-c362-4308-ba17-b1f4449b8294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:11.158078 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3371ed-c362-4308-ba17-b1f4449b8294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:11.159698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3371ed-c362-4308-ba17-b1f4449b8294 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:11.175952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-6d3371ed-c362-4308-ba17-b1f4449b8294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:11.175952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:11.175952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:11.175952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:11.175952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:11.175952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:11.175952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:11.175952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:11.175952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:11.175952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:11.175952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:11.175952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:11.175952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:11.175952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:11.175952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:11.176927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:11.176927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:11.176927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:11.176927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:11.176927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:11.176927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:11.176927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:11.176927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:11.176927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:11.176927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:11.176927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:11.176927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:11.176927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:11.176927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:11.176927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:11.176927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:11.176927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:11.178035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:11.178035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:11.178035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:11.178035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:11.178035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 330/653] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:11.178939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3371ed-c362-4308-ba17-b1f4449b8294 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:11.253630 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01430be-3923-4229-a8b1-406969f7cb19 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:11.293948 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec826339-6b4e-4295-ad94-45d803f24054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:11.294595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec826339-6b4e-4295-ad94-45d803f24054 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:11.294881 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec826339-6b4e-4295-ad94-45d803f24054 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:11.295160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec826339-6b4e-4295-ad94-45d803f24054 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:11.296019 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec826339-6b4e-4295-ad94-45d803f24054 None admin] RBAC: Adding query filter params (name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:11.296338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec826339-6b4e-4295-ad94-45d803f24054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:11.297780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec826339-6b4e-4295-ad94-45d803f24054 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:11.307752 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 324/654] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 08:58:11 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:11.308682 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec826339-6b4e-4295-ad94-45d803f24054 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:11.317169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3371ed-c362-4308-ba17-b1f4449b8294 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:11.326201 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:11.360647 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8655929-26f4-4868-b44a-8d12320254b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:11.361661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8655929-26f4-4868-b44a-8d12320254b1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:11.361946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8655929-26f4-4868-b44a-8d12320254b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:11.362221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8655929-26f4-4868-b44a-8d12320254b1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:11.371244 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8655929-26f4-4868-b44a-8d12320254b1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:11.372701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8655929-26f4-4868-b44a-8d12320254b1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:11.385914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-b8655929-26f4-4868-b44a-8d12320254b1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 02 08:58:11.385914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:11.385914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:11.385914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:11.385914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:11.385914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:11.385914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:11.385914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:11.385914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:11.385914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:11.385914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:11.385914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:11.385914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:11.385914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:11.385914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:11.385914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:11.386911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:11.386911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:11.386911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:11.386911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:11.386911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:11.386911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:11.386911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:11.386911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:11.386911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:11.386911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:11.386911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:11.386911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:11.386911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:11.386911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:11.386911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:11.386911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:11.386911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:11.386911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:11.387876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:11.387876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:11.387876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:11.387876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:11.387876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:11.387876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:11.387876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:11.387876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:11.387876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 02 08:58:11.387876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:11.387876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 331/655] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:58:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:11.388493 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8655929-26f4-4868-b44a-8d12320254b1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:11.396965 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:11.435891 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec826339-6b4e-4295-ad94-45d803f24054 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:11.475263 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552d4de-a823-4d87-bfa4-ffa8aa12726c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:11.475883 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552d4de-a823-4d87-bfa4-ffa8aa12726c None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:11.476163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552d4de-a823-4d87-bfa4-ffa8aa12726c None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:11.476436 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552d4de-a823-4d87-bfa4-ffa8aa12726c None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:11.477989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b552d4de-a823-4d87-bfa4-ffa8aa12726c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:11.478298 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b552d4de-a823-4d87-bfa4-ffa8aa12726c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:11.479666 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b552d4de-a823-4d87-bfa4-ffa8aa12726c None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:11.495057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 325/656] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:11.496187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552d4de-a823-4d87-bfa4-ffa8aa12726c None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:11.517181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8655929-26f4-4868-b44a-8d12320254b1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:11.557908 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd47e9a-8052-4fb2-8a23-e3cf0c9001c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:11.558609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd47e9a-8052-4fb2-8a23-e3cf0c9001c8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:11.558923 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd47e9a-8052-4fb2-8a23-e3cf0c9001c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:11.559175 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd47e9a-8052-4fb2-8a23-e3cf0c9001c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:11.570356 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd47e9a-8052-4fb2-8a23-e3cf0c9001c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:11.572329 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd47e9a-8052-4fb2-8a23-e3cf0c9001c8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:11.584735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:11.588662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-7dd47e9a-8052-4fb2-8a23-e3cf0c9001c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:11.588662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:11.588662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:11.588662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:11.588662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:11.588662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:11.588662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:11.588662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:11.588662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:11.588662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:11.588662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:11.588662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:11.588662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:11.588662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:11.588662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:11.589849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:11.589849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:11.589849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:11.589849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:11.589849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:11.589849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:11.589849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:11.589849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:11.589849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:11.589849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:11.589849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:11.589849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:11.589849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:11.589849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:11.589849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:11.589849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:11.589849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:11.590749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:11.590749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:11.590749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:11.590749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:11.590749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 332/657] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:11.591033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd47e9a-8052-4fb2-8a23-e3cf0c9001c8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:11.626991 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552d4de-a823-4d87-bfa4-ffa8aa12726c None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:11.655957 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:11.676689 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f8140-d817-447a-b87b-45f73d630177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:11.677396 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f8140-d817-447a-b87b-45f73d630177 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:11.678233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f8140-d817-447a-b87b-45f73d630177 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:11.678518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f8140-d817-447a-b87b-45f73d630177 None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:11.679318 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f8140-d817-447a-b87b-45f73d630177 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:11.679628 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f8140-d817-447a-b87b-45f73d630177 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:11.682146 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f8140-d817-447a-b87b-45f73d630177 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:11.705664 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd47e9a-8052-4fb2-8a23-e3cf0c9001c8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:11.706797 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 326/658] 10.222.0.100 () {56 vars in 1142 bytes} [Tue Aug 2 08:58:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:11.707975 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f8140-d817-447a-b87b-45f73d630177 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:11.743984 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d7666e-5707-4686-9f67-6a62da7bf232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:11.744725 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7666e-5707-4686-9f67-6a62da7bf232 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:11.745092 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7666e-5707-4686-9f67-6a62da7bf232 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:11.745468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7666e-5707-4686-9f67-6a62da7bf232 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:11.753221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d7666e-5707-4686-9f67-6a62da7bf232 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:11.754769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d7666e-5707-4686-9f67-6a62da7bf232 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:11.767206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-17d7666e-5707-4686-9f67-6a62da7bf232 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:11.767206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:11.767206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:11.767206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:11.767206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:11.767206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:11.767206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:11.767206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:11.767206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:11.767206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:11.767206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:11.767206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:11.767206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:11.767206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:11.767206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:11.769015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:11.769015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:11.769015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:11.769015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:11.769015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:11.769015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:11.769015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:11.769015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:11.769015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:11.769015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:11.769015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:11.769015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:11.769015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:11.769015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:11.769015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:11.769015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:11.769015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:11.770639 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:11.770639 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:11.770639 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:11.770639 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:11.770639 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:11.770639 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:11.770639 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:11.770639 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:11.774884 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 333/659] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:11.775844 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d7666e-5707-4686-9f67-6a62da7bf232 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:11.834336 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f8140-d817-447a-b87b-45f73d630177 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:11.864042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:11.875913 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192fea6-609b-410c-9a2d-c9b1deeaccf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:11.876757 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192fea6-609b-410c-9a2d-c9b1deeaccf4 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:11.877737 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192fea6-609b-410c-9a2d-c9b1deeaccf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:11.878174 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192fea6-609b-410c-9a2d-c9b1deeaccf4 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:11.879040 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7192fea6-609b-410c-9a2d-c9b1deeaccf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:11.879512 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7192fea6-609b-410c-9a2d-c9b1deeaccf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:11.881957 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7192fea6-609b-410c-9a2d-c9b1deeaccf4 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:11.899801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 327/660] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:11.900883 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192fea6-609b-410c-9a2d-c9b1deeaccf4 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:11.909651 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d7666e-5707-4686-9f67-6a62da7bf232 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:11.929643 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:11.951216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf16da2-170d-40fc-84fe-ef9207c230d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:11.951993 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf16da2-170d-40fc-84fe-ef9207c230d9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:11.952337 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf16da2-170d-40fc-84fe-ef9207c230d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:11.952718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf16da2-170d-40fc-84fe-ef9207c230d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:12.026957 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf16da2-170d-40fc-84fe-ef9207c230d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:12.028538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf16da2-170d-40fc-84fe-ef9207c230d9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:12.040329 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192fea6-609b-410c-9a2d-c9b1deeaccf4 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:12.089505 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a1678-2279-44a3-9db6-6bc0768da4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:12.090654 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a1678-2279-44a3-9db6-6bc0768da4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:12.091332 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a1678-2279-44a3-9db6-6bc0768da4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:12.091760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a1678-2279-44a3-9db6-6bc0768da4e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:12.093368 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a1678-2279-44a3-9db6-6bc0768da4e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:12.093839 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a1678-2279-44a3-9db6-6bc0768da4e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:12.096407 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a1678-2279-44a3-9db6-6bc0768da4e9 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:12.118879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 328/661] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:12.119795 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a1678-2279-44a3-9db6-6bc0768da4e9 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:12.145807 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 334/662] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:58:11 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:58:12.147172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf16da2-170d-40fc-84fe-ef9207c230d9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:12.173517 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:12.178975 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:12.233287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a1678-2279-44a3-9db6-6bc0768da4e9 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:12.270108 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf16da2-170d-40fc-84fe-ef9207c230d9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:12.273591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4eb46-2a50-4f72-91db-f93b9ea14042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:12.274464 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4eb46-2a50-4f72-91db-f93b9ea14042 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:12.274984 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4eb46-2a50-4f72-91db-f93b9ea14042 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:12.275461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4eb46-2a50-4f72-91db-f93b9ea14042 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:12.283663 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4eb46-2a50-4f72-91db-f93b9ea14042 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:12.285656 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4eb46-2a50-4f72-91db-f93b9ea14042 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:12.299109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-1bb4eb46-2a50-4f72-91db-f93b9ea14042 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 08:58:12.299109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:12.299109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:12.299109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:12.299109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:12.299109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:12.299109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:12.299109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:12.299109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:12.299109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:12.299109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:12.299109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:12.299109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:12.299109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:12.299109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:12.299109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:12.300339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:12.300339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:12.300339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:12.300339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:12.300339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:12.300339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:12.300339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:12.300339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:12.300339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:12.300339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:12.300339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:12.300339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:12.300339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:12.300339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:12.300339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:12.300339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:12.300339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:12.300339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:12.301554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:12.301554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:12.301554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:12.301554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:12.301554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:12.301554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:12.301554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:12.301554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:12.301554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 08:58:12.301554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:12.303496 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 329/663] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:58:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:12.304403 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4eb46-2a50-4f72-91db-f93b9ea14042 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:12.310311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016c1e5-8872-4b03-aebb-6740ec608b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:12.310943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016c1e5-8872-4b03-aebb-6740ec608b70 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:12.311225 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016c1e5-8872-4b03-aebb-6740ec608b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:12.311497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016c1e5-8872-4b03-aebb-6740ec608b70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:12.312279 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016c1e5-8872-4b03-aebb-6740ec608b70 None admin] RBAC: Adding query filter params (role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=8e55bb6b624c45a3a08306403570177d) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:12.312588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016c1e5-8872-4b03-aebb-6740ec608b70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:12.314081 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016c1e5-8872-4b03-aebb-6740ec608b70 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:12.333520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 335/664] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:58:12 2022] GET /identity/v3/role_assignments?user.id=8e55bb6b624c45a3a08306403570177d&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 610 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:12.334812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57666a-6411-4d81-bc67-2a56826bdf7a None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:12.335317 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57666a-6411-4d81-bc67-2a56826bdf7a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:12.335452 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57666a-6411-4d81-bc67-2a56826bdf7a None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:12.341148 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 336/665] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:12.341148 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57666a-6411-4d81-bc67-2a56826bdf7a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:12.438679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4eb46-2a50-4f72-91db-f93b9ea14042 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:12.449728 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:12.474282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:12.492808 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafe5ec-fd12-4349-badf-a92677e3707b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:12.494067 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafe5ec-fd12-4349-badf-a92677e3707b None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:12.494365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafe5ec-fd12-4349-badf-a92677e3707b None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:12.494644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafe5ec-fd12-4349-badf-a92677e3707b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:12.505060 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57666a-6411-4d81-bc67-2a56826bdf7a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:12.514098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eafe5ec-fd12-4349-badf-a92677e3707b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:12.515589 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eafe5ec-fd12-4349-badf-a92677e3707b None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:12.536725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-6eafe5ec-fd12-4349-badf-a92677e3707b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:12.536725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:12.536725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:12.536725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:12.536725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:12.536725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:12.536725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:12.536725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:12.536725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:12.536725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:12.536725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:12.536725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:12.536725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:12.536725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:12.536725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:12.537717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:12.537717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:12.537717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:12.537717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:12.537717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:12.537717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:12.537717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:12.537717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:12.537717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:12.537717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:12.537717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:12.537717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:12.537717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:12.537717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:12.537717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:12.537717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:12.537717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:12.540642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:12.540642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:12.540642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:12.540642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:12.540642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 330/666] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:12.540642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16941a-b879-442b-b710-1499974ce669 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:12.542380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16941a-b879-442b-b710-1499974ce669 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:12.542576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16941a-b879-442b-b710-1499974ce669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:12.561642 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54853d28-ee40-45f8-adcd-b69eadf973bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:12.562324 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54853d28-ee40-45f8-adcd-b69eadf973bb None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:12.563186 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54853d28-ee40-45f8-adcd-b69eadf973bb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:12.563186 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54853d28-ee40-45f8-adcd-b69eadf973bb None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:12.563393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-7d16941a-b879-442b-b710-1499974ce669 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:12.563640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54853d28-ee40-45f8-adcd-b69eadf973bb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:12.563950 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54853d28-ee40-45f8-adcd-b69eadf973bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:12.564446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-7d16941a-b879-442b-b710-1499974ce669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:12.565408 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54853d28-ee40-45f8-adcd-b69eadf973bb None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:12.583732 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 337/667] 10.222.0.100 () {56 vars in 1144 bytes} [Tue Aug 2 08:58:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:12.584463 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54853d28-ee40-45f8-adcd-b69eadf973bb None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:12.645180 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 331/668] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:12 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:58:12.646191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16941a-b879-442b-b710-1499974ce669 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:12.717866 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54853d28-ee40-45f8-adcd-b69eadf973bb None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:12.745768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:12.757334 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16941a-b879-442b-b710-1499974ce669 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:12.765056 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef343cfa-b433-47f9-8dd5-34d412f0da54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:12.765917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef343cfa-b433-47f9-8dd5-34d412f0da54 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:12.766883 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef343cfa-b433-47f9-8dd5-34d412f0da54 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:12.767311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef343cfa-b433-47f9-8dd5-34d412f0da54 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:12.768174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef343cfa-b433-47f9-8dd5-34d412f0da54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:12.769070 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef343cfa-b433-47f9-8dd5-34d412f0da54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:12.771045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef343cfa-b433-47f9-8dd5-34d412f0da54 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:12.778970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:12.789735 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 338/669] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:12.789735 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef343cfa-b433-47f9-8dd5-34d412f0da54 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:12.816649 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1a546-475f-4bdd-ab49-b9b1a11c8f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:12.818181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1a546-475f-4bdd-ab49-b9b1a11c8f01 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:12.818181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1a546-475f-4bdd-ab49-b9b1a11c8f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:12.818181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1a546-475f-4bdd-ab49-b9b1a11c8f01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:12.908357 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1a546-475f-4bdd-ab49-b9b1a11c8f01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:12.911204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1a546-475f-4bdd-ab49-b9b1a11c8f01 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:12.925646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef343cfa-b433-47f9-8dd5-34d412f0da54 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:12.971964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad6b74-049b-4ebf-bcbe-a916968f8f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:12.973132 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad6b74-049b-4ebf-bcbe-a916968f8f1e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:12.973539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad6b74-049b-4ebf-bcbe-a916968f8f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:12.973924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad6b74-049b-4ebf-bcbe-a916968f8f1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:12.989638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad6b74-049b-4ebf-bcbe-a916968f8f1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:12.998012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-bdad6b74-049b-4ebf-bcbe-a916968f8f1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:12.998012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:12.998012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:12.998012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:12.998012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:12.998012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:12.998012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:12.998012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:12.998012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:12.998012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:12.998012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:12.998012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:12.998012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:12.998012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:12.998012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:12.998012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:12.999293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:12.999293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:12.999293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:12.999293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:12.999293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:12.999293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:12.999293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:12.999293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:12.999293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:12.999293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:12.999293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:12.999293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:12.999293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:12.999293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:12.999293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:12.999293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:13.000219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:13.000219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:13.000219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:13.000219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:13.000219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:13.000219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:13.000219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:13.000219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:13.000219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:13.000219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:13.000219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:13.000219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:13.006445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 339/670] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:58:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:13.007550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad6b74-049b-4ebf-bcbe-a916968f8f1e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:13.037587 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:13.050796 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 332/671] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:58:12 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:58:13.051951 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1a546-475f-4bdd-ab49-b9b1a11c8f01 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:13.078734 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:13.126059 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad6b74-049b-4ebf-bcbe-a916968f8f1e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:13.158750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1a546-475f-4bdd-ab49-b9b1a11c8f01 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:13.162813 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8408a82-492f-4e60-aa68-cd8b42bd428e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:13.163313 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8408a82-492f-4e60-aa68-cd8b42bd428e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:13.163607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8408a82-492f-4e60-aa68-cd8b42bd428e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:13.163939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8408a82-492f-4e60-aa68-cd8b42bd428e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:13.171934 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8408a82-492f-4e60-aa68-cd8b42bd428e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:13.173424 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8408a82-492f-4e60-aa68-cd8b42bd428e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:13.186766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-f8408a82-492f-4e60-aa68-cd8b42bd428e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:13.186766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:13.186766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:13.186766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:13.186766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:13.186766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:13.186766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:13.186766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:13.186766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:13.186766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:13.186766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:13.186766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:13.186766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:13.186766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:13.186766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:13.187751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:13.187751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:13.187751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:13.187751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:13.187751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:13.187751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:13.187751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:13.187751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:13.187751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:13.187751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:13.187751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:13.187751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:13.187751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:13.187751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:13.187751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:13.187751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:13.187751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:13.188672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:13.188672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:13.188672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:13.188672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:13.188672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:13.188672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:13.188672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:13.188672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:13.192892 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 340/672] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:13.193756 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8408a82-492f-4e60-aa68-cd8b42bd428e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:13.203117 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88abb222-21cb-4375-827f-0b449fce037e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:13.204054 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abb222-21cb-4375-827f-0b449fce037e None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:13.204563 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abb222-21cb-4375-827f-0b449fce037e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:13.205049 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abb222-21cb-4375-827f-0b449fce037e None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:13.206003 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88abb222-21cb-4375-827f-0b449fce037e None admin] RBAC: Adding query filter params (name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:13.206522 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88abb222-21cb-4375-827f-0b449fce037e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:13.208108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88abb222-21cb-4375-827f-0b449fce037e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:13.217003 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 333/673] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 08:58:13 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:13.218314 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88abb222-21cb-4375-827f-0b449fce037e None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:13.275594 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:13.304014 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8408a82-492f-4e60-aa68-cd8b42bd428e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:13.322363 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:13.327358 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88abb222-21cb-4375-827f-0b449fce037e None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:13.342313 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9217b26-dbcd-49b5-b431-404f53b165c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:13.343183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9217b26-dbcd-49b5-b431-404f53b165c3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:13.343679 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9217b26-dbcd-49b5-b431-404f53b165c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:13.344183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9217b26-dbcd-49b5-b431-404f53b165c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:13.356070 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9217b26-dbcd-49b5-b431-404f53b165c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:13.357817 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9217b26-dbcd-49b5-b431-404f53b165c3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:13.365054 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbd023-b537-4fb6-aad1-cb481e114f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:13.365778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbd023-b537-4fb6-aad1-cb481e114f31 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:13.366176 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbd023-b537-4fb6-aad1-cb481e114f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:13.366526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbd023-b537-4fb6-aad1-cb481e114f31 None admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:13.368119 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fbd023-b537-4fb6-aad1-cb481e114f31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:13.368282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fbd023-b537-4fb6-aad1-cb481e114f31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:13.369875 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fbd023-b537-4fb6-aad1-cb481e114f31 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:13.375221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-e9217b26-dbcd-49b5-b431-404f53b165c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:13.375221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:13.375221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:13.375221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:13.375221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:13.375221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:13.375221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:13.375221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:13.375221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:13.375221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:13.375221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:13.375221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:13.375221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:13.375221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:13.375221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:13.376382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:13.376382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:13.376382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:13.376382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:13.376382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:13.376382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:13.376382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:13.376382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:13.376382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:13.376382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:13.376382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:13.376382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:13.376382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:13.376382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:13.376382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:13.376382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:13.376382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:13.377316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:13.377316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:13.377316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:13.377316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:13.383982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 341/674] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:13.385134 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9217b26-dbcd-49b5-b431-404f53b165c3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:13.388858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 334/675] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:13.389768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbd023-b537-4fb6-aad1-cb481e114f31 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:13.495642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbd023-b537-4fb6-aad1-cb481e114f31 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:13.496084 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9217b26-dbcd-49b5-b431-404f53b165c3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:13.510231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:13.531645 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c484d67d-9b7f-4e81-9fd3-d0b408dcea8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:13.532226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484d67d-9b7f-4e81-9fd3-d0b408dcea8f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:13.532515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484d67d-9b7f-4e81-9fd3-d0b408dcea8f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:13.532796 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484d67d-9b7f-4e81-9fd3-d0b408dcea8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:13.534873 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ca148-df0e-414c-9cd8-782469e6d9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:13.535487 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ca148-df0e-414c-9cd8-782469e6d9bf None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:13.535773 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ca148-df0e-414c-9cd8-782469e6d9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:13.536051 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ca148-df0e-414c-9cd8-782469e6d9bf None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:13.536740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ca148-df0e-414c-9cd8-782469e6d9bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:13.537046 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ca148-df0e-414c-9cd8-782469e6d9bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:13.538449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ca148-df0e-414c-9cd8-782469e6d9bf None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:13.545998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c484d67d-9b7f-4e81-9fd3-d0b408dcea8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:13.547509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c484d67d-9b7f-4e81-9fd3-d0b408dcea8f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:13.557140 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 335/676] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:13.557140 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ca148-df0e-414c-9cd8-782469e6d9bf None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:13.566397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-c484d67d-9b7f-4e81-9fd3-d0b408dcea8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:13.566397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:13.566397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:13.566397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:13.566397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:13.566397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:13.566397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:13.566397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:13.566397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:13.566397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:13.566397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:13.566397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:13.566397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:13.566397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:13.566397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:13.567618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:13.567618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:13.567618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:13.567618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:13.567618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:13.567618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:13.567618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:13.567618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:13.567618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:13.567618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:13.567618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:13.567618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:13.567618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:13.567618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:13.567618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:13.567618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:13.567618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:13.568534 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:13.568534 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:13.568534 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:13.568534 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:13.568534 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 342/677] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:13.568984 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec4c36-e40f-41cf-8d92-acd591966e0c None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:13.569166 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec4c36-e40f-41cf-8d92-acd591966e0c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:13.569497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec4c36-e40f-41cf-8d92-acd591966e0c None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:13.571244 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 343/678] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:13.572027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec4c36-e40f-41cf-8d92-acd591966e0c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:13.597917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:13.669895 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ca148-df0e-414c-9cd8-782469e6d9bf None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:13.682547 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec4c36-e40f-41cf-8d92-acd591966e0c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:13.706053 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4eefbe-8a81-4885-a2ac-51d3b58f0778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:13.706752 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4eefbe-8a81-4885-a2ac-51d3b58f0778 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:13.707110 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4eefbe-8a81-4885-a2ac-51d3b58f0778 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:13.707451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4eefbe-8a81-4885-a2ac-51d3b58f0778 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:13.708297 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4eefbe-8a81-4885-a2ac-51d3b58f0778 None admin] RBAC: Adding query filter params (role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=d959ddd087724547a11c3d4c796d8ab2) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:13.708698 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4eefbe-8a81-4885-a2ac-51d3b58f0778 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:13.710241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4eefbe-8a81-4885-a2ac-51d3b58f0778 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:13.721143 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 336/679] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:58:13 2022] GET /identity/v3/role_assignments?user.id=d959ddd087724547a11c3d4c796d8ab2&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 610 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:13.721143 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4eefbe-8a81-4885-a2ac-51d3b58f0778 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:13.729151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7603f44-bd6f-453a-a28d-fddb16e769ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:13.729151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7603f44-bd6f-453a-a28d-fddb16e769ed None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:13.729151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7603f44-bd6f-453a-a28d-fddb16e769ed None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:13.729151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7603f44-bd6f-453a-a28d-fddb16e769ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:13.732835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7603f44-bd6f-453a-a28d-fddb16e769ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:13.734579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7603f44-bd6f-453a-a28d-fddb16e769ed None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:13.748334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-a7603f44-bd6f-453a-a28d-fddb16e769ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:13.748334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:13.748334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:13.748334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:13.748334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:13.748334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:13.748334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:13.748334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:13.748334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:13.748334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:13.748334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:13.748334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:13.748334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:13.748334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:13.748334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:13.749512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:13.749512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:13.749512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:13.749512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:13.749512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:13.749512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:13.749512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:13.749512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:13.749512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:13.749512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:13.749512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:13.749512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:13.749512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:13.749512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:13.749512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:13.749512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:13.749512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:13.750873 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:13.750873 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:13.750873 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:13.750873 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:13.750873 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:13.750873 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:13.750873 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:13.750873 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:13.752260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 344/680] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:13.755334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83219d16-46db-48f6-9afa-26fdce797112 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:13.755334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83219d16-46db-48f6-9afa-26fdce797112 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:13.755334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83219d16-46db-48f6-9afa-26fdce797112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:13.774664 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-83219d16-46db-48f6-9afa-26fdce797112 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:13.776012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-83219d16-46db-48f6-9afa-26fdce797112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:13.778695 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:13.852935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 345/681] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:13 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:58:13.853380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4eefbe-8a81-4885-a2ac-51d3b58f0778 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:13.854164 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83219d16-46db-48f6-9afa-26fdce797112 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:13.869247 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:13.900053 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0adf5-fe4a-4a15-a179-dea3978b8973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:13.900852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0adf5-fe4a-4a15-a179-dea3978b8973 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:13.901273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0adf5-fe4a-4a15-a179-dea3978b8973 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:13.901672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0adf5-fe4a-4a15-a179-dea3978b8973 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:13.902995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0adf5-fe4a-4a15-a179-dea3978b8973 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:13.903427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0adf5-fe4a-4a15-a179-dea3978b8973 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:13.905275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0adf5-fe4a-4a15-a179-dea3978b8973 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:13.929064 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 337/682] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:13.929712 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0adf5-fe4a-4a15-a179-dea3978b8973 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:13.997710 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83219d16-46db-48f6-9afa-26fdce797112 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:14.043482 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e96ead-f53b-463c-99e5-e209dfcae297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:14.044139 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e96ead-f53b-463c-99e5-e209dfcae297 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:14.044422 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e96ead-f53b-463c-99e5-e209dfcae297 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:14.044773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e96ead-f53b-463c-99e5-e209dfcae297 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:14.046231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e96ead-f53b-463c-99e5-e209dfcae297 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:14.046231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e96ead-f53b-463c-99e5-e209dfcae297 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:14.047289 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e96ead-f53b-463c-99e5-e209dfcae297 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:14.058004 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:14.064377 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 346/683] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:14.065143 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e96ead-f53b-463c-99e5-e209dfcae297 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:14.074125 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0adf5-fe4a-4a15-a179-dea3978b8973 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:14.114542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0d9b6-4e84-419c-ac1c-b219e6057435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:14.115035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0d9b6-4e84-419c-ac1c-b219e6057435 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:14.115318 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0d9b6-4e84-419c-ac1c-b219e6057435 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:14.115590 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0d9b6-4e84-419c-ac1c-b219e6057435 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:14.131068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0d9b6-4e84-419c-ac1c-b219e6057435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:14.139825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-3af0d9b6-4e84-419c-ac1c-b219e6057435 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:58:14.139825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:14.139825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:14.139825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:14.139825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:14.139825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:14.139825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:14.139825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:14.139825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:14.139825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:14.139825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:14.139825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:14.139825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:14.139825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:14.139825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:14.139825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:14.140964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:14.140964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:14.140964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:14.140964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:14.140964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:14.140964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:14.140964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:14.140964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:14.140964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:14.140964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:14.140964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:14.140964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:14.140964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:14.140964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:14.140964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:14.140964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:14.141996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:14.141996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:14.141996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:14.141996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:14.141996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:14.141996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:14.141996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:14.141996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:14.141996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:14.141996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:14.141996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:58:14.141996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:14.145033 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 338/684] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:58:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:14.145997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0d9b6-4e84-419c-ac1c-b219e6057435 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:14.161491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:14.202365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e96ead-f53b-463c-99e5-e209dfcae297 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:14.251428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dac9c-1627-4d78-92d0-adec50bdf6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:14.252321 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dac9c-1627-4d78-92d0-adec50bdf6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:14.252770 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dac9c-1627-4d78-92d0-adec50bdf6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:14.253250 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dac9c-1627-4d78-92d0-adec50bdf6a5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:14.262579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588dac9c-1627-4d78-92d0-adec50bdf6a5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:14.264183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588dac9c-1627-4d78-92d0-adec50bdf6a5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:14.268806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0d9b6-4e84-419c-ac1c-b219e6057435 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:14.276318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-588dac9c-1627-4d78-92d0-adec50bdf6a5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:58:14.276318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:14.276318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:14.276318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:14.276318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:14.276318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:14.276318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:14.276318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:14.276318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:14.276318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:14.276318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:14.276318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:14.276318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:14.276318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:14.276318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:14.276318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:14.277516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:14.277516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:14.277516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:14.277516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:14.277516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:14.277516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:14.277516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:14.277516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:14.277516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:14.277516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:14.277516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:14.277516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:14.277516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:14.277516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:14.277516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:14.277516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:14.277516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:14.277516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:14.278665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:14.278665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:14.278665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:14.278665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:14.278665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:14.278665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:14.278665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:14.278665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:14.278665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:58:14.278665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:14.280105 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 347/685] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:58:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:14.281003 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dac9c-1627-4d78-92d0-adec50bdf6a5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:14.316528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2ef62-fbc9-4269-aae8-873e689055f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:14.317298 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2ef62-fbc9-4269-aae8-873e689055f0 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:14.317651 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2ef62-fbc9-4269-aae8-873e689055f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:14.317995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2ef62-fbc9-4269-aae8-873e689055f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:14.318857 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2ef62-fbc9-4269-aae8-873e689055f0 None admin] RBAC: Adding query filter params (role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=f4afef33deb247509765a8c88a78ac39) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:14.319240 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2ef62-fbc9-4269-aae8-873e689055f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:14.320837 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2ef62-fbc9-4269-aae8-873e689055f0 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:14.327600 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 339/686] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:58:14 2022] GET /identity/v3/role_assignments?user.id=f4afef33deb247509765a8c88a78ac39&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 610 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:14.328623 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2ef62-fbc9-4269-aae8-873e689055f0 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:14.354699 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:14.419769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dac9c-1627-4d78-92d0-adec50bdf6a5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:14.449022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2ef62-fbc9-4269-aae8-873e689055f0 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:14.463716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:14.472027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cecc5f-5290-4eb4-845a-514c957f676f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:14.472655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cecc5f-5290-4eb4-845a-514c957f676f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:14.472938 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cecc5f-5290-4eb4-845a-514c957f676f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:14.477812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cecc5f-5290-4eb4-845a-514c957f676f None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:14.477812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cecc5f-5290-4eb4-845a-514c957f676f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:14.477812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cecc5f-5290-4eb4-845a-514c957f676f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:14.477812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cecc5f-5290-4eb4-845a-514c957f676f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:14.495382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 348/687] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:58:14 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:14.502276 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cecc5f-5290-4eb4-845a-514c957f676f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:14.514099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b831c-5fb1-495b-898e-c0f8c5d96d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:14.514733 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b831c-5fb1-495b-898e-c0f8c5d96d4b None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:14.515011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b831c-5fb1-495b-898e-c0f8c5d96d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:14.515281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b831c-5fb1-495b-898e-c0f8c5d96d4b None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:14.516081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b831c-5fb1-495b-898e-c0f8c5d96d4b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:14.516395 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b831c-5fb1-495b-898e-c0f8c5d96d4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:14.520048 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b831c-5fb1-495b-898e-c0f8c5d96d4b None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:14.562263 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 340/688] 10.222.0.100 () {56 vars in 1140 bytes} [Tue Aug 2 08:58:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:14.568794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b831c-5fb1-495b-898e-c0f8c5d96d4b None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:14.664022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cecc5f-5290-4eb4-845a-514c957f676f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:14.665443 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:14.688969 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b831c-5fb1-495b-898e-c0f8c5d96d4b None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:14.707174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2501ef-9bf1-4bb5-a2d0-6653e01edbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:14.708198 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2501ef-9bf1-4bb5-a2d0-6653e01edbcb None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:14.708496 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2501ef-9bf1-4bb5-a2d0-6653e01edbcb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:14.708774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2501ef-9bf1-4bb5-a2d0-6653e01edbcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:14.723927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2501ef-9bf1-4bb5-a2d0-6653e01edbcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:14.725419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2501ef-9bf1-4bb5-a2d0-6653e01edbcb None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:14.728167 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956b40b-bdfb-4f43-b986-abd5f8097a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:14.729009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956b40b-bdfb-4f43-b986-abd5f8097a24 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:14.729551 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956b40b-bdfb-4f43-b986-abd5f8097a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:14.730000 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956b40b-bdfb-4f43-b986-abd5f8097a24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:14.740554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-bb2501ef-9bf1-4bb5-a2d0-6653e01edbcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:14.740554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:14.740554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:14.740554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:14.740554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:14.740554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:14.740554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:14.740554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:14.740554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:14.740554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:14.740554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:14.740554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:14.740554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:14.740554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:14.740554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:14.741523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:14.741523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:14.741523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:14.741523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:14.741523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:14.741523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:14.741523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:14.741523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:14.741523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:14.741523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:14.741523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:14.741523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:14.741523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:14.741523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:14.741523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:14.741523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:14.741523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:14.743155 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:14.743155 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:14.743155 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:14.743155 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:14.747133 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 349/689] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:14.748461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd4ddc-ac64-4768-81fb-4e7675a00e12 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:14.748753 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd4ddc-ac64-4768-81fb-4e7675a00e12 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:14.749034 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd4ddc-ac64-4768-81fb-4e7675a00e12 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:14.751097 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 350/690] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:14.751821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd4ddc-ac64-4768-81fb-4e7675a00e12 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:14.809828 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956b40b-bdfb-4f43-b986-abd5f8097a24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:14.811264 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956b40b-bdfb-4f43-b986-abd5f8097a24 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:14.819844 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:14.896066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd4ddc-ac64-4768-81fb-4e7675a00e12 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:14.915565 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:14.944217 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d1a0f-52e9-44da-867e-a7611620c264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:14.944955 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 341/691] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:58:14 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:58:14.945823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d1a0f-52e9-44da-867e-a7611620c264 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:14.946334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d1a0f-52e9-44da-867e-a7611620c264 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:14.946764 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d1a0f-52e9-44da-867e-a7611620c264 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:14.947199 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973409a7-1b99-4c52-839d-525da98d7bab None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:14.947577 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973409a7-1b99-4c52-839d-525da98d7bab None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:14.947952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d1a0f-52e9-44da-867e-a7611620c264 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:14.948070 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973409a7-1b99-4c52-839d-525da98d7bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:14.948504 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d1a0f-52e9-44da-867e-a7611620c264 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:14.950046 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d1a0f-52e9-44da-867e-a7611620c264 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:14.965582 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 351/692] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:14.966709 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d1a0f-52e9-44da-867e-a7611620c264 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:14.968659 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-973409a7-1b99-4c52-839d-525da98d7bab None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:14.970159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-973409a7-1b99-4c52-839d-525da98d7bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:15.053132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 342/693] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:58:15.053132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973409a7-1b99-4c52-839d-525da98d7bab None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:15.111551 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d1a0f-52e9-44da-867e-a7611620c264 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:15.120822 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:15.160631 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0460f-9576-4e14-af45-a6f459e90c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:15.161850 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0460f-9576-4e14-af45-a6f459e90c25 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:15.162133 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0460f-9576-4e14-af45-a6f459e90c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:15.162404 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0460f-9576-4e14-af45-a6f459e90c25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:15.175602 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0460f-9576-4e14-af45-a6f459e90c25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:15.177325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0460f-9576-4e14-af45-a6f459e90c25 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:15.192153 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973409a7-1b99-4c52-839d-525da98d7bab None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:15.192649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-70a0460f-9576-4e14-af45-a6f459e90c25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:15.192649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:15.192649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:15.192649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:15.192649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:15.192649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:15.192649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:15.192649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:15.192649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:15.192649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:15.192649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:15.192649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:15.192649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:15.192649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:15.192649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:15.193595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:15.193595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:15.193595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:15.193595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:15.193595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:15.193595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:15.193595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:15.193595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:15.193595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:15.193595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:15.193595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:15.193595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:15.193595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:15.193595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:15.193595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:15.193595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:15.193595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:15.194746 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:15.194746 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:15.194746 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:15.194746 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:15.196422 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 352/694] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:15.197315 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0460f-9576-4e14-af45-a6f459e90c25 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:15.207648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:15.247421 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823bf60c-1bd9-4a53-b350-168541c13dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:15.248955 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bf60c-1bd9-4a53-b350-168541c13dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:15.249434 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bf60c-1bd9-4a53-b350-168541c13dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:15.249878 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bf60c-1bd9-4a53-b350-168541c13dd2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:15.261146 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823bf60c-1bd9-4a53-b350-168541c13dd2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:15.263502 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823bf60c-1bd9-4a53-b350-168541c13dd2 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:15.280295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-823bf60c-1bd9-4a53-b350-168541c13dd2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:58:15.280295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:15.280295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:15.280295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:15.280295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:15.280295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:15.280295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:15.280295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:15.280295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:15.280295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:15.280295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:15.280295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:15.280295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:15.280295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:15.280295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:15.280295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:15.281485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:15.281485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:15.281485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:15.281485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:15.281485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:15.281485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:15.281485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:15.281485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:15.281485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:15.281485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:15.281485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:15.281485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:15.281485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:15.281485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:15.281485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:15.281485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:15.281485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:15.281485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:15.282677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:15.282677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:15.282677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:15.282677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:15.282677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:15.282677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:15.282677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:15.282677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:15.282677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:58:15.282677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:15.284601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 343/695] 10.222.0.100 () {56 vars in 1098 bytes} [Tue Aug 2 08:58:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:15.285558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823bf60c-1bd9-4a53-b350-168541c13dd2 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:15.312898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0460f-9576-4e14-af45-a6f459e90c25 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:15.351390 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28751030-e73a-42da-a84f-13aeb67c108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:15.352731 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28751030-e73a-42da-a84f-13aeb67c108c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:15.353188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28751030-e73a-42da-a84f-13aeb67c108c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:15.353632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28751030-e73a-42da-a84f-13aeb67c108c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:15.354678 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28751030-e73a-42da-a84f-13aeb67c108c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:15.356617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28751030-e73a-42da-a84f-13aeb67c108c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:15.373520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-28751030-e73a-42da-a84f-13aeb67c108c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 08:58:15.373520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:15.373520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:15.373520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:15.373520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:15.373520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:15.373520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:15.373520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:15.373520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 08:58:15.373520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:15.373520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:15.373520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:15.373520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:15.373520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:15.373520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:15.373520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:15.374718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:15.374718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:15.374718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:15.374718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:15.374718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:15.374718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:15.374718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:15.374718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:15.374718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:15.374718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:15.374718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:15.374718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:15.374718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:15.374718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:15.374718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:15.374718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:15.376027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:15.376027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:15.376027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:15.376027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:15.376027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:15.376027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:15.376027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:15.376027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:15.376027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:15.376027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:15.376027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:15.376027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:15.376027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:15.376027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 08:58:15.376027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:15.383784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 353/696] 10.222.0.100 () {56 vars in 1134 bytes} [Tue Aug 2 08:58:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:15.383784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28751030-e73a-42da-a84f-13aeb67c108c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:15.416777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:15.418979 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823bf60c-1bd9-4a53-b350-168541c13dd2 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:15.457321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da6f34-0d76-44ef-8846-ccfe416c451c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:15.457995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da6f34-0d76-44ef-8846-ccfe416c451c None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:15.458356 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da6f34-0d76-44ef-8846-ccfe416c451c None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:15.458671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da6f34-0d76-44ef-8846-ccfe416c451c None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:15.459447 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da6f34-0d76-44ef-8846-ccfe416c451c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:15.459835 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da6f34-0d76-44ef-8846-ccfe416c451c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:15.461657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da6f34-0d76-44ef-8846-ccfe416c451c None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:15.474886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 344/697] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:15.475851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da6f34-0d76-44ef-8846-ccfe416c451c None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:15.506948 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:15.517952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28751030-e73a-42da-a84f-13aeb67c108c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:15.561915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20930237-f190-4773-a388-e4c244ecffab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:15.562897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20930237-f190-4773-a388-e4c244ecffab None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:15.563715 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20930237-f190-4773-a388-e4c244ecffab None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:15.564176 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20930237-f190-4773-a388-e4c244ecffab None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:15.565494 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20930237-f190-4773-a388-e4c244ecffab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:15.566286 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20930237-f190-4773-a388-e4c244ecffab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:15.568072 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20930237-f190-4773-a388-e4c244ecffab None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:15.588805 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 354/698] 10.222.0.100 () {56 vars in 1138 bytes} [Tue Aug 2 08:58:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:15.589815 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20930237-f190-4773-a388-e4c244ecffab None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:15.595471 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da6f34-0d76-44ef-8846-ccfe416c451c None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:15.639674 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88997d8a-f1d1-4ca1-960f-011d35c396c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:15.641007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88997d8a-f1d1-4ca1-960f-011d35c396c4 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:15.641634 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88997d8a-f1d1-4ca1-960f-011d35c396c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:15.642189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88997d8a-f1d1-4ca1-960f-011d35c396c4 None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:15.643211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88997d8a-f1d1-4ca1-960f-011d35c396c4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:15.643705 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88997d8a-f1d1-4ca1-960f-011d35c396c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:15.645598 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88997d8a-f1d1-4ca1-960f-011d35c396c4 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:15.652257 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 345/699] 10.222.0.100 () {56 vars in 1129 bytes} [Tue Aug 2 08:58:15 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:15.653403 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88997d8a-f1d1-4ca1-960f-011d35c396c4 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:15.690772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:15.717379 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20930237-f190-4773-a388-e4c244ecffab None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:15.762094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:15.767545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c622afb-3912-43a7-9e2a-f5cf023c1876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:15.768651 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c622afb-3912-43a7-9e2a-f5cf023c1876 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:15.770006 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c622afb-3912-43a7-9e2a-f5cf023c1876 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:15.770906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c622afb-3912-43a7-9e2a-f5cf023c1876 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:15.772723 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88997d8a-f1d1-4ca1-960f-011d35c396c4 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:15.779928 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c622afb-3912-43a7-9e2a-f5cf023c1876 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:15.782237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c622afb-3912-43a7-9e2a-f5cf023c1876 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:15.796983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-2c622afb-3912-43a7-9e2a-f5cf023c1876 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:15.796983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:15.796983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:15.796983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:15.796983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:15.796983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:15.796983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:15.796983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:15.796983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:15.796983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:15.796983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:15.796983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:15.796983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:15.796983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:15.796983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:15.797963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:15.797963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:15.797963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:15.797963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:15.797963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:15.797963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:15.797963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:15.797963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:15.797963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:15.797963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:15.797963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:15.797963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:15.797963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:15.797963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:15.797963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:15.797963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:15.797963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:15.798875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:15.798875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:15.798875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:15.798875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:15.798875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:15.798875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:15.798875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:15.798875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:15.798875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 355/700] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:15.800549 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c622afb-3912-43a7-9e2a-f5cf023c1876 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:15.817067 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b10ea-8002-4183-ab41-55e7f6922c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:15.817826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b10ea-8002-4183-ab41-55e7f6922c22 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:15.818211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b10ea-8002-4183-ab41-55e7f6922c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:15.818554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b10ea-8002-4183-ab41-55e7f6922c22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:15.891329 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b10ea-8002-4183-ab41-55e7f6922c22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:15.892885 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b10ea-8002-4183-ab41-55e7f6922c22 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:15.938429 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c622afb-3912-43a7-9e2a-f5cf023c1876 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:15.989782 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0cf23-8da1-46c6-b130-5c8d9a4331ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:15.990932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0cf23-8da1-46c6-b130-5c8d9a4331ef None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:15.992608 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0cf23-8da1-46c6-b130-5c8d9a4331ef None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:15.993644 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0cf23-8da1-46c6-b130-5c8d9a4331ef None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:15.994419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0cf23-8da1-46c6-b130-5c8d9a4331ef None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:15.994798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0cf23-8da1-46c6-b130-5c8d9a4331ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:15.996237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0cf23-8da1-46c6-b130-5c8d9a4331ef None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:16.004732 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:16.008726 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 356/701] 10.222.0.100 () {56 vars in 1129 bytes} [Tue Aug 2 08:58:15 2022] GET /identity/v3/services?type=block-storage => generated 131 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:16.009818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0cf23-8da1-46c6-b130-5c8d9a4331ef None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:16.019144 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:16.020430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 346/702] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:58:15 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:58:16.026702 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b10ea-8002-4183-ab41-55e7f6922c22 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:16.156933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0cf23-8da1-46c6-b130-5c8d9a4331ef None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:16.161775 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b10ea-8002-4183-ab41-55e7f6922c22 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:16.200091 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9e7c4-f604-43c2-976a-b85bb9dc2983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:16.200723 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9e7c4-f604-43c2-976a-b85bb9dc2983 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:16.201011 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9e7c4-f604-43c2-976a-b85bb9dc2983 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:16.201318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9e7c4-f604-43c2-976a-b85bb9dc2983 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:16.202076 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9e7c4-f604-43c2-976a-b85bb9dc2983 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:16.202384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9e7c4-f604-43c2-976a-b85bb9dc2983 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:16.202662 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50437707-bc9b-46e2-96fd-ed382a703d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:16.203460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50437707-bc9b-46e2-96fd-ed382a703d9f None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:16.203826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9e7c4-f604-43c2-976a-b85bb9dc2983 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:16.204108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50437707-bc9b-46e2-96fd-ed382a703d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:16.204529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50437707-bc9b-46e2-96fd-ed382a703d9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:16.218429 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50437707-bc9b-46e2-96fd-ed382a703d9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:16.220124 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50437707-bc9b-46e2-96fd-ed382a703d9f None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:16.224301 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 357/703] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:16.225891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69905baf-31d3-4e16-828d-142c053e9adf None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:16.226187 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69905baf-31d3-4e16-828d-142c053e9adf None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:16.227092 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69905baf-31d3-4e16-828d-142c053e9adf None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:16.228749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 358/704] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:16 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:16.234257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1700f33-8711-42cb-a783-d6a821b15299 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:16.234547 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1700f33-8711-42cb-a783-d6a821b15299 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:16.234816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1700f33-8711-42cb-a783-d6a821b15299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:16.240192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-50437707-bc9b-46e2-96fd-ed382a703d9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:16.240192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:16.240192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:16.240192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:16.240192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:16.240192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:16.240192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:16.240192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:16.240192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:16.240192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:16.240192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:16.240192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:16.240192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:16.240192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:16.240192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:16.241286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:16.241286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:16.241286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:16.241286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:16.241286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:16.241286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:16.241286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:16.241286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:16.241286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:16.241286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:16.241286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:16.241286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:16.241286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:16.241286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:16.241286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:16.241286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:16.241286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:16.243198 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:16.243198 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:16.243198 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:16.243198 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:16.243198 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 347/705] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:16.243806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50437707-bc9b-46e2-96fd-ed382a703d9f None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:16.265026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a1700f33-8711-42cb-a783-d6a821b15299 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:16.266257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a1700f33-8711-42cb-a783-d6a821b15299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:16.309148 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:16.331737 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:16.341294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 359/706] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:16 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:58:16.342437 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1700f33-8711-42cb-a783-d6a821b15299 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:16.392756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50437707-bc9b-46e2-96fd-ed382a703d9f None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:16.438980 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e299090-8a75-4ef4-99f7-f0d730e99247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:16.439956 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e299090-8a75-4ef4-99f7-f0d730e99247 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:16.440460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e299090-8a75-4ef4-99f7-f0d730e99247 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:16.440929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e299090-8a75-4ef4-99f7-f0d730e99247 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:16.441939 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e299090-8a75-4ef4-99f7-f0d730e99247 None admin] RBAC: Adding query filter params (role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=50cb532b58ed4951ba960a0a7d579b49) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:16.442448 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e299090-8a75-4ef4-99f7-f0d730e99247 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:16.444092 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e299090-8a75-4ef4-99f7-f0d730e99247 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:16.451671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 348/707] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:58:16 2022] GET /identity/v3/role_assignments?user.id=50cb532b58ed4951ba960a0a7d579b49&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 610 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:16.452714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e299090-8a75-4ef4-99f7-f0d730e99247 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:16.501798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1700f33-8711-42cb-a783-d6a821b15299 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:16.573044 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10100ee2-9d0b-43b7-a453-9e9e32a78617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:16.573725 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10100ee2-9d0b-43b7-a453-9e9e32a78617 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:16.574013 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10100ee2-9d0b-43b7-a453-9e9e32a78617 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:16.574291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10100ee2-9d0b-43b7-a453-9e9e32a78617 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:16.575015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10100ee2-9d0b-43b7-a453-9e9e32a78617 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:16.575326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10100ee2-9d0b-43b7-a453-9e9e32a78617 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:16.576725 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10100ee2-9d0b-43b7-a453-9e9e32a78617 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:16.595023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 360/708] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:16.600577 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10100ee2-9d0b-43b7-a453-9e9e32a78617 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:16.666813 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:16.677827 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e299090-8a75-4ef4-99f7-f0d730e99247 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:16.691310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:16.724053 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10100ee2-9d0b-43b7-a453-9e9e32a78617 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:16.724598 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c445400-1f89-4c4b-927d-45e261280f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:16.725049 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c445400-1f89-4c4b-927d-45e261280f4b None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:16.725341 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c445400-1f89-4c4b-927d-45e261280f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:16.725620 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c445400-1f89-4c4b-927d-45e261280f4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:16.738786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c445400-1f89-4c4b-927d-45e261280f4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:16.740350 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c445400-1f89-4c4b-927d-45e261280f4b None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:16.758412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-4c445400-1f89-4c4b-927d-45e261280f4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:16.758412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:16.758412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:16.758412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:16.758412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:16.758412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:16.758412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:16.758412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:16.758412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:16.758412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:16.758412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:16.758412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:16.758412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:16.758412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:16.758412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:16.759517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:16.759517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:16.759517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:16.759517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:16.759517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:16.759517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:16.759517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:16.759517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:16.759517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:16.759517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:16.759517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:16.759517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:16.759517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:16.759517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:16.759517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:16.759517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:16.759517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:16.760455 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:16.760455 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:16.760455 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:16.760455 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:16.763926 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 349/709] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:16.765335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cdd51-35b1-47ac-a3f3-27a774d419bd None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:16.765852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cdd51-35b1-47ac-a3f3-27a774d419bd None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:16.765960 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cdd51-35b1-47ac-a3f3-27a774d419bd None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:16.767536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 350/710] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:16.768637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cdd51-35b1-47ac-a3f3-27a774d419bd None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:16.775168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d75bc2-6d29-4f5d-af35-f720f5303f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:16.776069 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d75bc2-6d29-4f5d-af35-f720f5303f95 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:16.776411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d75bc2-6d29-4f5d-af35-f720f5303f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:16.776696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d75bc2-6d29-4f5d-af35-f720f5303f95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:16.786359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d75bc2-6d29-4f5d-af35-f720f5303f95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:16.787996 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d75bc2-6d29-4f5d-af35-f720f5303f95 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:16.807088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-03d75bc2-6d29-4f5d-af35-f720f5303f95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:16.807088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:16.807088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:16.807088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:16.807088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:16.807088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:16.807088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:16.807088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:16.807088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:16.807088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:16.807088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:16.807088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:16.807088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:16.807088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:16.807088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:16.808043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:16.808043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:16.808043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:16.808043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:16.808043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:16.808043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:16.808043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:16.808043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:16.808043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:16.808043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:16.808043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:16.808043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:16.808043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:16.808043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:16.808043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:16.808043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:16.808043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:16.808933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:16.808933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:16.808933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:16.808933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:16.808933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:16.808933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:16.808933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:16.808933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:16.810592 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 361/711] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:16.814840 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b418796-1de1-4e6d-84b9-bd8f557eaead None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:16.814840 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b418796-1de1-4e6d-84b9-bd8f557eaead None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:16.814840 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b418796-1de1-4e6d-84b9-bd8f557eaead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:16.843812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9b418796-1de1-4e6d-84b9-bd8f557eaead None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:16.845949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9b418796-1de1-4e6d-84b9-bd8f557eaead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:16.891091 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cdd51-35b1-47ac-a3f3-27a774d419bd None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:16.936122 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef436cff-fbfd-4bd6-a269-6f20bd17e28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:16.936836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef436cff-fbfd-4bd6-a269-6f20bd17e28c None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:16.937229 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef436cff-fbfd-4bd6-a269-6f20bd17e28c None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:16.937583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef436cff-fbfd-4bd6-a269-6f20bd17e28c None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:16.938341 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef436cff-fbfd-4bd6-a269-6f20bd17e28c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:16.938719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef436cff-fbfd-4bd6-a269-6f20bd17e28c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:16.940169 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef436cff-fbfd-4bd6-a269-6f20bd17e28c None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:16.957204 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 362/712] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:16 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:58:16.957204 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b418796-1de1-4e6d-84b9-bd8f557eaead None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:16.963666 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 351/713] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:16.967479 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef436cff-fbfd-4bd6-a269-6f20bd17e28c None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:16.995231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:17.025591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:17.099978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b418796-1de1-4e6d-84b9-bd8f557eaead None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:17.100177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef436cff-fbfd-4bd6-a269-6f20bd17e28c None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:17.145716 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09288ba6-77a2-48b0-a6bf-59e8646cdc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:17.146341 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09288ba6-77a2-48b0-a6bf-59e8646cdc3f None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:17.146629 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09288ba6-77a2-48b0-a6bf-59e8646cdc3f None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:17.146910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09288ba6-77a2-48b0-a6bf-59e8646cdc3f None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:17.147594 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09288ba6-77a2-48b0-a6bf-59e8646cdc3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:17.149096 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09288ba6-77a2-48b0-a6bf-59e8646cdc3f None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:17.150061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd1908-c99f-4324-898d-938eb0714fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:17.150972 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd1908-c99f-4324-898d-938eb0714fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:17.151456 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd1908-c99f-4324-898d-938eb0714fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:17.152031 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd1908-c99f-4324-898d-938eb0714fd6 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:17.153020 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd1908-c99f-4324-898d-938eb0714fd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:17.153578 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd1908-c99f-4324-898d-938eb0714fd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:17.155177 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd1908-c99f-4324-898d-938eb0714fd6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:17.178742 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 352/715] 10.222.0.100 () {60 vars in 1137 bytes} [Tue Aug 2 08:58:16 2022] POST /identity/v3/users => generated 255 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:58:17.178742 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09288ba6-77a2-48b0-a6bf-59e8646cdc3f None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:17.179264 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 363/715] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:17.181496 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd1908-c99f-4324-898d-938eb0714fd6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:17.290438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:17.346004 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09288ba6-77a2-48b0-a6bf-59e8646cdc3f None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:17.377420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd1908-c99f-4324-898d-938eb0714fd6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:17.385440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:17.426610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ca07f-1d0b-496c-9ca8-dbc3d82b0d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:17.427275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ca07f-1d0b-496c-9ca8-dbc3d82b0d60 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:17.427543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ca07f-1d0b-496c-9ca8-dbc3d82b0d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:17.427811 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ca07f-1d0b-496c-9ca8-dbc3d82b0d60 None admin] PATH_INFO: `/v3/services/network` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:17.428443 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ca07f-1d0b-496c-9ca8-dbc3d82b0d60 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:17.431193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ca07f-1d0b-496c-9ca8-dbc3d82b0d60 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:17.440985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae2ec1-27c8-45fa-964b-f1797842cb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:17.441960 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae2ec1-27c8-45fa-964b-f1797842cb10 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:17.442497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae2ec1-27c8-45fa-964b-f1797842cb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:17.442969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae2ec1-27c8-45fa-964b-f1797842cb10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:17.443380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-ff1ca07f-1d0b-496c-9ca8-dbc3d82b0d60 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 08:58:17.443380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:17.443380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:17.443380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:17.443380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:17.443380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:17.443380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:17.443380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:17.443380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 08:58:17.443380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:17.443380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:17.443380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:17.443380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:17.443380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:17.443380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:17.443380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:17.443380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:17.444344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:17.444344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:17.444344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:17.444344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:17.444344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:17.444344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:17.444344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:17.444344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:17.444344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:17.444344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:17.444344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:17.444344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:17.444344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:17.444344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:17.444344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:17.444344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:17.445671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:17.445671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:17.445671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:17.445671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:17.445671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:17.445671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:17.445671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:17.445671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:17.445671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:17.445671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:17.445671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:17.445671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:17.445671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 08:58:17.445671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:17.447777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae2ec1-27c8-45fa-964b-f1797842cb10 None admin] RBAC: Adding query filter params (role.id=7cbff51867a7475d837afd51f6c12f11, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=49a0c1a4a6cf43e28ec1b8488c681848) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:17.448317 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae2ec1-27c8-45fa-964b-f1797842cb10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:17.449244 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 353/716] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:17 2022] GET /identity/v3/services/network => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:17.450173 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae2ec1-27c8-45fa-964b-f1797842cb10 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:17.457477 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 364/717] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:58:17 2022] GET /identity/v3/role_assignments?user.id=49a0c1a4a6cf43e28ec1b8488c681848&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=7cbff51867a7475d837afd51f6c12f11 => generated 260 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:17.573076 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ca07f-1d0b-496c-9ca8-dbc3d82b0d60 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:17.687548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ca07f-1d0b-496c-9ca8-dbc3d82b0d60 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:17.724157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce710a5-6c3b-4207-ad7c-0f28d89b09e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:17.724785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce710a5-6c3b-4207-ad7c-0f28d89b09e4 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:17.725065 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce710a5-6c3b-4207-ad7c-0f28d89b09e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:17.725366 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce710a5-6c3b-4207-ad7c-0f28d89b09e4 None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:17.726052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce710a5-6c3b-4207-ad7c-0f28d89b09e4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:17.726924 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce710a5-6c3b-4207-ad7c-0f28d89b09e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:17.728316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce710a5-6c3b-4207-ad7c-0f28d89b09e4 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:17.737215 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 354/718] 10.222.0.100 () {56 vars in 1117 bytes} [Tue Aug 2 08:58:17 2022] GET /identity/v3/services?name=network => generated 125 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:17.738856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae2ec1-27c8-45fa-964b-f1797842cb10 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:17.846065 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae2ec1-27c8-45fa-964b-f1797842cb10 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:17.895749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c695b36-c9c4-4ce9-8fae-d69a2adc134d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:17.896384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c695b36-c9c4-4ce9-8fae-d69a2adc134d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:17.896672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c695b36-c9c4-4ce9-8fae-d69a2adc134d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:17.896950 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c695b36-c9c4-4ce9-8fae-d69a2adc134d None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:17.897673 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c695b36-c9c4-4ce9-8fae-d69a2adc134d None admin] RBAC: Adding query filter params (type=network) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:17.897982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c695b36-c9c4-4ce9-8fae-d69a2adc134d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:17.899993 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c695b36-c9c4-4ce9-8fae-d69a2adc134d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:17.910607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 365/719] 10.222.0.100 () {56 vars in 1117 bytes} [Tue Aug 2 08:58:17 2022] GET /identity/v3/services?type=network => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:18.157861 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522b022-d1ad-48da-a030-ae58411ddc6e None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:18.158383 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522b022-d1ad-48da-a030-ae58411ddc6e None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:18.158383 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522b022-d1ad-48da-a030-ae58411ddc6e None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:18.160139 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 355/720] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:18.166313 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad95db-5754-44d6-9466-a5958658e47f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:18.166603 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad95db-5754-44d6-9466-a5958658e47f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:18.166881 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad95db-5754-44d6-9466-a5958658e47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:18.187830 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-69ad95db-5754-44d6-9466-a5958658e47f None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:18.188992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-69ad95db-5754-44d6-9466-a5958658e47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:18.220354 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:18.270111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 366/721] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:58:18.300463 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522b022-d1ad-48da-a030-ae58411ddc6e None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:18.306227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:18.437559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522b022-d1ad-48da-a030-ae58411ddc6e None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:18.479962 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c38d94-5c72-4d3d-8226-ee34adb3e869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:18.480593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c38d94-5c72-4d3d-8226-ee34adb3e869 None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:18.480878 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c38d94-5c72-4d3d-8226-ee34adb3e869 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:18.481166 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c38d94-5c72-4d3d-8226-ee34adb3e869 None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:18.481935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c38d94-5c72-4d3d-8226-ee34adb3e869 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:18.483298 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c38d94-5c72-4d3d-8226-ee34adb3e869 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:18.509344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 356/722] 10.222.0.100 () {60 vars in 1146 bytes} [Tue Aug 2 08:58:18 2022] POST /identity/v3/services => generated 252 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:58:19.129698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b0d76-e717-499c-b0f4-9c65933645ef None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:19.130207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b0d76-e717-499c-b0f4-9c65933645ef None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:19.130311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b0d76-e717-499c-b0f4-9c65933645ef None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:19.131865 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 367/723] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:19.137266 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e638d4-025b-4fef-a760-6dede6ace3ba None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:19.137554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e638d4-025b-4fef-a760-6dede6ace3ba None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:19.137828 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e638d4-025b-4fef-a760-6dede6ace3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:19.162075 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-31e638d4-025b-4fef-a760-6dede6ace3ba None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:19.163272 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-31e638d4-025b-4fef-a760-6dede6ace3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:19.251385 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:19.271794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 357/724] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:19 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:58:19.314551 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b0d76-e717-499c-b0f4-9c65933645ef None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:19.462460 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b0d76-e717-499c-b0f4-9c65933645ef None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:19.515248 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18356d06-649f-4604-a04e-37a6e7887000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:19.515910 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18356d06-649f-4604-a04e-37a6e7887000 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:19.516196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18356d06-649f-4604-a04e-37a6e7887000 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:19.516475 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18356d06-649f-4604-a04e-37a6e7887000 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:19.531330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18356d06-649f-4604-a04e-37a6e7887000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:19.536350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-18356d06-649f-4604-a04e-37a6e7887000 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:58:19.536350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:19.536350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:19.536350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:19.536350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:19.536350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:19.536350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:19.536350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:19.536350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:19.536350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:19.536350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:19.536350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:19.536350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:19.536350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:19.536350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:19.536350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:19.537353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:19.537353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:19.537353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:19.537353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:19.537353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:19.537353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:19.537353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:19.537353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:19.537353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:19.537353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:19.537353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:19.537353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:19.537353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:19.537353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:19.537353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:19.537353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:19.538228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:19.538228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:19.538228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:19.538228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:19.538228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:19.538228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:19.538228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:19.538228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:19.538228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:19.538228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:19.538228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:58:19.538228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:19.538228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 368/725] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:58:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:19.544890 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e638d4-025b-4fef-a760-6dede6ace3ba None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:19.712790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e638d4-025b-4fef-a760-6dede6ace3ba None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:19.752898 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe91384-5a06-4598-8d0d-f00a153f092a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:19.753687 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe91384-5a06-4598-8d0d-f00a153f092a None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:19.753963 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe91384-5a06-4598-8d0d-f00a153f092a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:19.754234 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe91384-5a06-4598-8d0d-f00a153f092a None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:19.754947 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe91384-5a06-4598-8d0d-f00a153f092a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:19.755248 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe91384-5a06-4598-8d0d-f00a153f092a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:19.756606 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe91384-5a06-4598-8d0d-f00a153f092a None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:19.765786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 358/726] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:58:19 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:19.771272 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18356d06-649f-4604-a04e-37a6e7887000 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:19.938451 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:19.950593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429b3be-4799-4454-aa2e-5c7cbfb48098 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:19.950870 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429b3be-4799-4454-aa2e-5c7cbfb48098 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:19.951145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429b3be-4799-4454-aa2e-5c7cbfb48098 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:19.952730 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 359/727] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:19.957637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ce02f-7018-497c-afc3-d0282831a1f5 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:19.957916 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ce02f-7018-497c-afc3-d0282831a1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:19.958218 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ce02f-7018-497c-afc3-d0282831a1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:19.979071 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-3f5ce02f-7018-497c-afc3-d0282831a1f5 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:19.980564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3f5ce02f-7018-497c-afc3-d0282831a1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:20.039790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:20.046850 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18356d06-649f-4604-a04e-37a6e7887000 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:20.068384 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 360/728] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:19 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:58:20.069800 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c5f18-5206-4fde-99ad-a2ba80379642 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:20.070623 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c5f18-5206-4fde-99ad-a2ba80379642 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:20.070899 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c5f18-5206-4fde-99ad-a2ba80379642 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:20.072442 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 361/729] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:20.077507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec5e4e-1001-41d1-b349-218ca9fe3986 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:20.077787 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec5e4e-1001-41d1-b349-218ca9fe3986 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:20.078063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec5e4e-1001-41d1-b349-218ca9fe3986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:20.099403 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-60ec5e4e-1001-41d1-b349-218ca9fe3986 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:20.100578 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-60ec5e4e-1001-41d1-b349-218ca9fe3986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:20.108454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d6685-ffe0-411a-89d8-6cd4ce1df295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:20.109817 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d6685-ffe0-411a-89d8-6cd4ce1df295 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:20.110100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d6685-ffe0-411a-89d8-6cd4ce1df295 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:20.110372 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d6685-ffe0-411a-89d8-6cd4ce1df295 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:20.128278 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d6685-ffe0-411a-89d8-6cd4ce1df295 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:20.129788 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d6685-ffe0-411a-89d8-6cd4ce1df295 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:20.145943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-7c7d6685-ffe0-411a-89d8-6cd4ce1df295 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:20.145943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:20.145943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:20.145943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:20.145943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:20.145943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:20.145943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:20.145943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:20.145943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:20.145943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:20.145943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:20.145943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:20.145943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:20.145943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:20.145943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:20.146906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:20.146906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:20.146906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:20.146906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:20.146906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:20.146906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:20.146906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:20.146906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:20.146906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:20.146906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:20.146906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:20.146906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:20.146906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:20.146906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:20.146906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:20.146906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:20.146906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:20.147826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:20.147826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:20.147826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:20.147826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:20.151831 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 369/730] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:20.152932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d6685-ffe0-411a-89d8-6cd4ce1df295 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:20.196145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 362/731] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:20 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:58:20.198229 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec5e4e-1001-41d1-b349-218ca9fe3986 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:20.233815 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:20.273889 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d6685-ffe0-411a-89d8-6cd4ce1df295 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:20.322981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef5029-718a-4c16-b5e4-805ca340ecfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:20.324508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef5029-718a-4c16-b5e4-805ca340ecfe None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:20.325009 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef5029-718a-4c16-b5e4-805ca340ecfe None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:20.325770 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef5029-718a-4c16-b5e4-805ca340ecfe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:20.339852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec5e4e-1001-41d1-b349-218ca9fe3986 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:20.345235 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef5029-718a-4c16-b5e4-805ca340ecfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:20.352218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-4cef5029-718a-4c16-b5e4-805ca340ecfe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:20.352218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:20.352218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:20.352218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:20.352218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:20.352218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:20.352218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:20.352218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:20.352218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:20.352218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:20.352218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:20.352218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:20.352218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:20.352218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:20.352218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:20.352218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:20.353231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:20.353231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:20.353231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:20.353231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:20.353231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:20.353231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:20.353231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:20.353231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:20.353231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:20.353231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:20.353231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:20.353231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:20.353231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:20.353231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:20.353231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:20.353231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:20.354213 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:20.354213 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:20.354213 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:20.354213 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:20.354213 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:20.354213 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:20.354213 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:20.354213 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:20.354213 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:20.354213 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:20.354213 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:20.354213 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:20.355803 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:20.361250 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 370/732] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:58:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:20.362321 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef5029-718a-4c16-b5e4-805ca340ecfe None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:20.386769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5cb2b-fe3b-4457-91ca-25b649b30241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:20.387507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5cb2b-fe3b-4457-91ca-25b649b30241 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:20.388240 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5cb2b-fe3b-4457-91ca-25b649b30241 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:20.388597 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5cb2b-fe3b-4457-91ca-25b649b30241 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:20.389404 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5cb2b-fe3b-4457-91ca-25b649b30241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:20.389784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5cb2b-fe3b-4457-91ca-25b649b30241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:20.391242 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5cb2b-fe3b-4457-91ca-25b649b30241 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:20.405267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 363/733] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:20 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:20.406190 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5cb2b-fe3b-4457-91ca-25b649b30241 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:20.495446 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef5029-718a-4c16-b5e4-805ca340ecfe None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:20.518343 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5cb2b-fe3b-4457-91ca-25b649b30241 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:20.533211 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16ac88-8e9a-45c6-ac37-1cc2d8b0f5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:20.533855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16ac88-8e9a-45c6-ac37-1cc2d8b0f5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:20.534141 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16ac88-8e9a-45c6-ac37-1cc2d8b0f5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:20.534419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16ac88-8e9a-45c6-ac37-1cc2d8b0f5a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:20.536358 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:20.548002 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f16ac88-8e9a-45c6-ac37-1cc2d8b0f5a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:20.553033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-7f16ac88-8e9a-45c6-ac37-1cc2d8b0f5a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:58:20.553033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:20.553033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:20.553033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:20.553033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:20.553033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:20.553033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:20.553033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:20.553033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:20.553033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:20.553033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:20.553033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:20.553033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:20.553033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:20.553033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:20.553033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:20.553981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:20.553981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:20.553981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:20.553981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:20.553981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:20.553981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:20.553981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:20.553981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:20.553981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:20.553981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:20.553981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:20.553981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:20.553981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:20.553981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:20.553981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:20.553981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:20.555987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:20.555987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:20.555987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:20.555987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:20.555987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:20.555987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:20.555987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:20.555987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:20.555987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:20.555987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:20.555987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:58:20.555987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:20.555987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 371/734] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:58:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:20.555987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16ac88-8e9a-45c6-ac37-1cc2d8b0f5a1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:20.558043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6093e19-1086-4917-b68d-dbe6b139be35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:20.558666 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6093e19-1086-4917-b68d-dbe6b139be35 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:20.558951 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6093e19-1086-4917-b68d-dbe6b139be35 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:20.559223 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6093e19-1086-4917-b68d-dbe6b139be35 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:20.560379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6093e19-1086-4917-b68d-dbe6b139be35 None admin] RBAC: Adding query filter params (name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:20.560687 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6093e19-1086-4917-b68d-dbe6b139be35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:20.562085 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6093e19-1086-4917-b68d-dbe6b139be35 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:20.576002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 364/735] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 08:58:20 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:20.576002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7b439-4fcf-4587-b76d-2f5038ad786c None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:20.576002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7b439-4fcf-4587-b76d-2f5038ad786c None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:20.576002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7b439-4fcf-4587-b76d-2f5038ad786c None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:20.578944 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 365/736] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:20 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:20.579711 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7b439-4fcf-4587-b76d-2f5038ad786c None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:20.618093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:20.671967 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16ac88-8e9a-45c6-ac37-1cc2d8b0f5a1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:20.690338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7b439-4fcf-4587-b76d-2f5038ad786c None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:20.708321 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a40bb-f7fd-431f-b9a3-463966fee81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:20.708968 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a40bb-f7fd-431f-b9a3-463966fee81c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:20.709290 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a40bb-f7fd-431f-b9a3-463966fee81c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:20.709569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a40bb-f7fd-431f-b9a3-463966fee81c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:20.717572 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a40bb-f7fd-431f-b9a3-463966fee81c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:20.718997 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a40bb-f7fd-431f-b9a3-463966fee81c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:20.727052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638c0f4-44eb-4854-8583-e1827677ca08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:20.727668 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638c0f4-44eb-4854-8583-e1827677ca08 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:20.728033 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638c0f4-44eb-4854-8583-e1827677ca08 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:20.728315 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638c0f4-44eb-4854-8583-e1827677ca08 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:20.729035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638c0f4-44eb-4854-8583-e1827677ca08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:20.729370 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638c0f4-44eb-4854-8583-e1827677ca08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:20.730726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638c0f4-44eb-4854-8583-e1827677ca08 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:20.731112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ea6a40bb-f7fd-431f-b9a3-463966fee81c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:58:20.731112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:20.731112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:20.731112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:20.731112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:20.731112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:20.731112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:20.731112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:20.731112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:20.731112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:20.731112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:20.731112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:20.731112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:20.731112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:20.731112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:20.731112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:20.732026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:20.732026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:20.732026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:20.732026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:20.732026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:20.732026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:20.732026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:20.732026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:20.732026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:20.732026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:20.732026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:20.732026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:20.732026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:20.732026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:20.732026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:20.732026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:20.732026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:20.732026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:20.733055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:20.733055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:20.733055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:20.733055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:20.733055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:20.733055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:20.733055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:20.733055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:20.733055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:58:20.733055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:20.733055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 372/737] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:58:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:20.733055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a40bb-f7fd-431f-b9a3-463966fee81c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:20.745428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 366/738] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:58:20 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:20.746681 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95daa871-f754-4564-a1b1-aac67ec26799 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:20.746975 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95daa871-f754-4564-a1b1-aac67ec26799 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:20.747253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95daa871-f754-4564-a1b1-aac67ec26799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:20.767533 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-95daa871-f754-4564-a1b1-aac67ec26799 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:20.769345 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-95daa871-f754-4564-a1b1-aac67ec26799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:20.796344 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:20.846459 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 367/739] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:20 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:58:20.846866 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a40bb-f7fd-431f-b9a3-463966fee81c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:20.847468 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95daa871-f754-4564-a1b1-aac67ec26799 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:20.875745 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:20.896586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae82e9a-f5b6-47d3-b1ac-bf00f8de8288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:20.897265 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae82e9a-f5b6-47d3-b1ac-bf00f8de8288 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:20.897555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae82e9a-f5b6-47d3-b1ac-bf00f8de8288 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:20.897838 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae82e9a-f5b6-47d3-b1ac-bf00f8de8288 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:20.909492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae82e9a-f5b6-47d3-b1ac-bf00f8de8288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:20.910936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae82e9a-f5b6-47d3-b1ac-bf00f8de8288 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:20.933495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-bae82e9a-f5b6-47d3-b1ac-bf00f8de8288 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:20.933495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:20.933495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:20.933495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:20.933495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:20.933495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:20.933495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:20.933495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:20.933495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:20.933495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:20.933495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:20.933495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:20.933495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:20.933495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:20.933495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:20.934444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:20.934444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:20.934444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:20.934444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:20.934444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:20.934444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:20.934444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:20.934444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:20.934444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:20.934444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:20.934444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:20.934444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:20.934444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:20.934444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:20.934444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:20.934444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:20.934444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:20.935333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:20.935333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:20.935333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:20.935333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:20.935333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 373/740] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:20.936051 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae82e9a-f5b6-47d3-b1ac-bf00f8de8288 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:20.970395 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95daa871-f754-4564-a1b1-aac67ec26799 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:21.012784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2452526-67cb-429c-91af-3ddb45c41990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:21.013493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2452526-67cb-429c-91af-3ddb45c41990 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:21.013777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2452526-67cb-429c-91af-3ddb45c41990 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:21.014051 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2452526-67cb-429c-91af-3ddb45c41990 None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:21.014842 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2452526-67cb-429c-91af-3ddb45c41990 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:21.015144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2452526-67cb-429c-91af-3ddb45c41990 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:21.016564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2452526-67cb-429c-91af-3ddb45c41990 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:21.033313 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 368/741] 10.222.0.100 () {56 vars in 1140 bytes} [Tue Aug 2 08:58:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:21.034099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2452526-67cb-429c-91af-3ddb45c41990 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:21.052788 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae82e9a-f5b6-47d3-b1ac-bf00f8de8288 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:21.060598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:21.090677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57dd04-d3e7-458f-9549-4626f00413ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:21.091341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57dd04-d3e7-458f-9549-4626f00413ac None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:21.091618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57dd04-d3e7-458f-9549-4626f00413ac None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:21.091895 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57dd04-d3e7-458f-9549-4626f00413ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:21.103369 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c57dd04-d3e7-458f-9549-4626f00413ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:21.104717 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c57dd04-d3e7-458f-9549-4626f00413ac None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:21.119585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-0c57dd04-d3e7-458f-9549-4626f00413ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:21.119585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:21.119585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:21.119585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:21.119585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:21.119585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:21.119585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:21.119585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:21.119585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:21.119585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:21.119585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:21.119585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:21.119585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:21.119585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:21.119585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:21.120559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:21.120559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:21.120559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:21.120559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:21.120559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:21.120559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:21.120559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:21.120559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:21.120559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:21.120559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:21.120559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:21.120559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:21.120559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:21.120559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:21.120559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:21.120559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:21.120559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:21.121703 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:21.121703 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:21.121703 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:21.121703 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:21.125568 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 374/742] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:21.126391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57dd04-d3e7-458f-9549-4626f00413ac None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:21.138039 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:21.157132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2452526-67cb-429c-91af-3ddb45c41990 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:21.193711 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5714989-3d64-4a6f-8d07-f8c28fab2a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:21.194481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5714989-3d64-4a6f-8d07-f8c28fab2a52 None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:21.194785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5714989-3d64-4a6f-8d07-f8c28fab2a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:21.195354 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5714989-3d64-4a6f-8d07-f8c28fab2a52 None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:21.196111 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5714989-3d64-4a6f-8d07-f8c28fab2a52 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:21.197429 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5714989-3d64-4a6f-8d07-f8c28fab2a52 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:21.217148 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 369/743] 10.222.0.100 () {60 vars in 1146 bytes} [Tue Aug 2 08:58:21 2022] POST /identity/v3/services => generated 241 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:58:21.217148 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5714989-3d64-4a6f-8d07-f8c28fab2a52 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:21.237547 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57dd04-d3e7-458f-9549-4626f00413ac None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:21.294031 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b5061-6a82-4e19-a438-9169f1de34ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:21.294666 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b5061-6a82-4e19-a438-9169f1de34ff None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:21.294957 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b5061-6a82-4e19-a438-9169f1de34ff None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:21.295231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b5061-6a82-4e19-a438-9169f1de34ff None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:21.295961 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b5061-6a82-4e19-a438-9169f1de34ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:21.296266 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b5061-6a82-4e19-a438-9169f1de34ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:21.297717 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b5061-6a82-4e19-a438-9169f1de34ff None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:21.311803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 375/744] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:21.312589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b5061-6a82-4e19-a438-9169f1de34ff None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:21.332239 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:21.340871 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5714989-3d64-4a6f-8d07-f8c28fab2a52 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:21.383409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d053169-a735-4aee-a549-c50b85f9adbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:21.384637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d053169-a735-4aee-a549-c50b85f9adbc None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:21.384921 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d053169-a735-4aee-a549-c50b85f9adbc None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:21.385233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d053169-a735-4aee-a549-c50b85f9adbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:21.401657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:21.436687 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b5061-6a82-4e19-a438-9169f1de34ff None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:21.459380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d053169-a735-4aee-a549-c50b85f9adbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:21.461159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d053169-a735-4aee-a549-c50b85f9adbc None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:21.473227 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be0e44-1940-40f2-a567-eda428377f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:21.473905 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be0e44-1940-40f2-a567-eda428377f6b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:21.474310 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be0e44-1940-40f2-a567-eda428377f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:21.474589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be0e44-1940-40f2-a567-eda428377f6b None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:21.475403 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be0e44-1940-40f2-a567-eda428377f6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:21.475717 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be0e44-1940-40f2-a567-eda428377f6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:21.477076 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be0e44-1940-40f2-a567-eda428377f6b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:21.495126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 376/745] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:21.496451 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9bf82-02a3-4cbc-acac-a42ad643cc4e None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:21.496739 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9bf82-02a3-4cbc-acac-a42ad643cc4e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:21.497015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9bf82-02a3-4cbc-acac-a42ad643cc4e None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:21.502431 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 377/746] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:21.502431 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9bf82-02a3-4cbc-acac-a42ad643cc4e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:21.569175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 370/747] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:58:21 2022] GET /identity/v3/auth/tokens => generated 1003 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:58:21.569175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d053169-a735-4aee-a549-c50b85f9adbc None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:21.588038 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:21.616378 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:21.622422 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9bf82-02a3-4cbc-acac-a42ad643cc4e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:21.659952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7603c9-c4e9-4a76-9529-1d05dbc57aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:21.660514 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7603c9-c4e9-4a76-9529-1d05dbc57aaf None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:21.660799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7603c9-c4e9-4a76-9529-1d05dbc57aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:21.661074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7603c9-c4e9-4a76-9529-1d05dbc57aaf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:21.669600 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7603c9-c4e9-4a76-9529-1d05dbc57aaf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:21.671042 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7603c9-c4e9-4a76-9529-1d05dbc57aaf None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:21.675995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d053169-a735-4aee-a549-c50b85f9adbc None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:21.684515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-8c7603c9-c4e9-4a76-9529-1d05dbc57aaf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:58:21.684515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:21.684515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:21.684515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:21.684515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:21.684515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:21.684515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:21.684515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:21.684515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:21.684515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:21.684515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:21.684515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:21.684515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:21.684515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:21.684515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:21.684515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:21.685698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:21.685698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:21.685698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:21.685698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:21.685698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:21.685698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:21.685698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:21.685698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:21.685698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:21.685698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:21.685698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:21.685698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:21.685698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:21.685698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:21.685698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:21.685698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:21.685698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:21.685698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:21.686684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:21.686684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:21.686684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:21.686684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:21.686684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:21.686684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:21.686684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:21.686684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:21.686684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:58:21.686684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:21.686684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 378/748] 10.222.0.100 () {56 vars in 1122 bytes} [Tue Aug 2 08:58:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:21.687367 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d09899-7253-4df1-8bcc-79935d8b225e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:21.687557 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d09899-7253-4df1-8bcc-79935d8b225e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:21.687842 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d09899-7253-4df1-8bcc-79935d8b225e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:21.705678 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-26d09899-7253-4df1-8bcc-79935d8b225e None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:21.706868 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-26d09899-7253-4df1-8bcc-79935d8b225e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:21.718985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ef84c-59f8-46b0-baab-bafdce814a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:21.719859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ef84c-59f8-46b0-baab-bafdce814a5c None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:21.720288 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ef84c-59f8-46b0-baab-bafdce814a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:21.720713 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ef84c-59f8-46b0-baab-bafdce814a5c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:21.729433 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ef84c-59f8-46b0-baab-bafdce814a5c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:21.730878 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ef84c-59f8-46b0-baab-bafdce814a5c None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:21.743108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-f86ef84c-59f8-46b0-baab-bafdce814a5c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:58:21.743108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:21.743108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:21.743108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:21.743108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:21.743108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:21.743108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:21.743108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:21.743108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:21.743108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:21.743108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:21.743108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:21.743108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:21.743108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:21.743108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:21.743108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:21.744083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:21.744083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:21.744083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:21.744083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:21.744083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:21.744083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:21.744083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:21.744083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:21.744083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:21.744083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:21.744083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:21.744083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:21.744083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:21.744083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:21.744083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:21.744083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:21.744083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:21.744083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:21.745026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:21.745026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:21.745026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:21.745026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:21.745026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:21.745026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:21.745026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:21.745026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:21.745026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:58:21.745026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:21.745026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 371/749] 10.222.0.100 () {56 vars in 1098 bytes} [Tue Aug 2 08:58:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:21.745026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ef84c-59f8-46b0-baab-bafdce814a5c None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:21.781935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 379/750] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:21 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:21.782784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d09899-7253-4df1-8bcc-79935d8b225e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:21.848656 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:21.874685 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ef84c-59f8-46b0-baab-bafdce814a5c None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:21.878147 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:21.888902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d09899-7253-4df1-8bcc-79935d8b225e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:21.919078 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649f32e-55bc-495a-a790-a74b959170ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:21.919664 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649f32e-55bc-495a-a790-a74b959170ac None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:21.919949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649f32e-55bc-495a-a790-a74b959170ac None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:21.920228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649f32e-55bc-495a-a790-a74b959170ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:21.931974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59de6e-54c5-45bf-baeb-d4ecf691d332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:21.932189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649f32e-55bc-495a-a790-a74b959170ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:21.932735 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59de6e-54c5-45bf-baeb-d4ecf691d332 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:21.933085 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59de6e-54c5-45bf-baeb-d4ecf691d332 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:21.933462 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59de6e-54c5-45bf-baeb-d4ecf691d332 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:21.933604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649f32e-55bc-495a-a790-a74b959170ac None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:21.934386 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59de6e-54c5-45bf-baeb-d4ecf691d332 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:21.934764 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59de6e-54c5-45bf-baeb-d4ecf691d332 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:21.936241 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59de6e-54c5-45bf-baeb-d4ecf691d332 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:21.948734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-4649f32e-55bc-495a-a790-a74b959170ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:21.948734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:21.948734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:21.948734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:21.948734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:21.948734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:21.948734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:21.948734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:21.948734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:21.948734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:21.948734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:21.948734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:21.948734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:21.948734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:21.948734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:21.950073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:21.950073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:21.950073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:21.950073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:21.950073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:21.950073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:21.950073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:21.950073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:21.950073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:21.950073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:21.950073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:21.950073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:21.950073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:21.950073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:21.950073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:21.950073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:21.950073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:21.951214 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:21.951214 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:21.951214 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:21.951214 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:21.954622 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 372/751] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:21.955386 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649f32e-55bc-495a-a790-a74b959170ac None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:21.960434 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 380/752] 10.222.0.100 () {56 vars in 1154 bytes} [Tue Aug 2 08:58:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:21.960434 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59de6e-54c5-45bf-baeb-d4ecf691d332 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:22.063696 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649f32e-55bc-495a-a790-a74b959170ac None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:22.065990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59de6e-54c5-45bf-baeb-d4ecf691d332 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:22.090349 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:22.099744 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ac6f7-e3fb-45b7-8172-f91381d66b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:22.100366 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ac6f7-e3fb-45b7-8172-f91381d66b56 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:22.100652 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ac6f7-e3fb-45b7-8172-f91381d66b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:22.100957 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ac6f7-e3fb-45b7-8172-f91381d66b56 None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:22.101783 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ac6f7-e3fb-45b7-8172-f91381d66b56 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:22.102101 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ac6f7-e3fb-45b7-8172-f91381d66b56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:22.103487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10541c-a733-4f22-9584-a715dbc7fa00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:22.103609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ac6f7-e3fb-45b7-8172-f91381d66b56 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:22.104322 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10541c-a733-4f22-9584-a715dbc7fa00 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:22.104699 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10541c-a733-4f22-9584-a715dbc7fa00 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:22.105077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10541c-a733-4f22-9584-a715dbc7fa00 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:22.105821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f10541c-a733-4f22-9584-a715dbc7fa00 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:22.107252 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f10541c-a733-4f22-9584-a715dbc7fa00 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:22.116208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-3f10541c-a733-4f22-9584-a715dbc7fa00 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 08:58:22.116208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:22.116208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:22.116208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:22.116208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:22.116208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:22.116208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:22.116208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:22.116208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 08:58:22.116208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:22.116208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:22.116208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:22.116208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:22.116208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:22.116208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:22.116208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:22.117897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:22.117897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:22.117897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:22.117897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:22.117897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:22.117897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:22.117897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:22.117897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:22.117897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:22.117897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:22.117897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:22.117897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:22.117897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:22.117897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:22.117897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:22.117897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:22.118913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:22.118913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:22.118913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:22.118913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:22.118913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:22.118913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:22.118913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:22.118913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:22.118913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:22.118913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:22.118913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:22.118913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:22.118913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:22.118913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 08:58:22.118913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:22.120315 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 381/753] 10.222.0.100 () {56 vars in 1134 bytes} [Tue Aug 2 08:58:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:22.121302 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10541c-a733-4f22-9584-a715dbc7fa00 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:22.133564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 373/754] 10.222.0.100 () {56 vars in 1138 bytes} [Tue Aug 2 08:58:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:22.133564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ac6f7-e3fb-45b7-8172-f91381d66b56 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:22.148129 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:22.227553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10541c-a733-4f22-9584-a715dbc7fa00 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:22.239406 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ac6f7-e3fb-45b7-8172-f91381d66b56 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:22.264469 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87174b8-6229-4eab-b3aa-b41bfbcf48a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:22.264989 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87174b8-6229-4eab-b3aa-b41bfbcf48a1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:22.265304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87174b8-6229-4eab-b3aa-b41bfbcf48a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:22.265579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87174b8-6229-4eab-b3aa-b41bfbcf48a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:22.266261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87174b8-6229-4eab-b3aa-b41bfbcf48a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:22.266560 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87174b8-6229-4eab-b3aa-b41bfbcf48a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:22.267900 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87174b8-6229-4eab-b3aa-b41bfbcf48a1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:22.282399 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb86ec-af69-4c2b-9bc5-12a2ed895413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:22.283013 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb86ec-af69-4c2b-9bc5-12a2ed895413 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:22.283291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb86ec-af69-4c2b-9bc5-12a2ed895413 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:22.283561 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb86ec-af69-4c2b-9bc5-12a2ed895413 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:22.287206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 382/755] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:22.288128 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87174b8-6229-4eab-b3aa-b41bfbcf48a1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:22.340388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:22.363513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb86ec-af69-4c2b-9bc5-12a2ed895413 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:22.364970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb86ec-af69-4c2b-9bc5-12a2ed895413 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:22.383185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:22.400204 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87174b8-6229-4eab-b3aa-b41bfbcf48a1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:22.434178 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ef6a2-8a7f-4c55-ac00-0576788c1fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:22.434794 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ef6a2-8a7f-4c55-ac00-0576788c1fae None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:22.435077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ef6a2-8a7f-4c55-ac00-0576788c1fae None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:22.435355 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ef6a2-8a7f-4c55-ac00-0576788c1fae None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:22.436028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ef6a2-8a7f-4c55-ac00-0576788c1fae None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:22.436338 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ef6a2-8a7f-4c55-ac00-0576788c1fae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:22.437724 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ef6a2-8a7f-4c55-ac00-0576788c1fae None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:22.442776 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 383/756] 10.222.0.100 () {56 vars in 1129 bytes} [Tue Aug 2 08:58:22 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:22.443551 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ef6a2-8a7f-4c55-ac00-0576788c1fae None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:22.481388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 374/757] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:58:22 2022] GET /identity/v3/auth/tokens => generated 1003 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:58:22.482520 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb86ec-af69-4c2b-9bc5-12a2ed895413 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:22.555225 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ef6a2-8a7f-4c55-ac00-0576788c1fae None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:22.565836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:22.586626 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb86ec-af69-4c2b-9bc5-12a2ed895413 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:22.591709 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dcff6c-52c1-4ed5-9039-ae395eb85ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:22.592353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcff6c-52c1-4ed5-9039-ae395eb85ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:22.592638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcff6c-52c1-4ed5-9039-ae395eb85ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:22.592917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcff6c-52c1-4ed5-9039-ae395eb85ab8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:22.602846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:22.603839 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dcff6c-52c1-4ed5-9039-ae395eb85ab8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:22.605591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dcff6c-52c1-4ed5-9039-ae395eb85ab8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:22.620535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-10dcff6c-52c1-4ed5-9039-ae395eb85ab8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:22.620535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:22.620535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:22.620535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:22.620535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:22.620535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:22.620535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:22.620535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:22.620535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:22.620535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:22.620535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:22.620535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:22.620535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:22.620535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:22.620535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:22.621591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:22.621591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:22.621591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:22.621591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:22.621591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:22.621591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:22.621591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:22.621591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:22.621591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:22.621591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:22.621591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:22.621591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:22.621591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:22.621591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:22.621591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:22.621591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:22.621591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:22.622654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:22.622654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:22.622654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:22.622654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:22.622654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 384/758] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:22.622931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dcff6c-52c1-4ed5-9039-ae395eb85ab8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:22.626705 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddb17b-9fb9-4b66-867b-92dd8ff8f00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:22.627384 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddb17b-9fb9-4b66-867b-92dd8ff8f00a None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:22.627666 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddb17b-9fb9-4b66-867b-92dd8ff8f00a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:22.627938 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddb17b-9fb9-4b66-867b-92dd8ff8f00a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:22.635365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddb17b-9fb9-4b66-867b-92dd8ff8f00a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:22.636798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddb17b-9fb9-4b66-867b-92dd8ff8f00a None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:22.648686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-7cddb17b-9fb9-4b66-867b-92dd8ff8f00a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:22.648686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:22.648686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:22.648686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:22.648686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:22.648686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:22.648686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:22.648686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:22.648686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:22.648686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:22.648686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:22.648686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:22.648686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:22.648686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:22.648686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:22.649762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:22.649762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:22.649762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:22.649762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:22.649762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:22.649762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:22.649762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:22.649762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:22.649762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:22.649762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:22.649762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:22.649762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:22.649762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:22.649762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:22.649762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:22.649762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:22.649762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:22.650852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:22.650852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:22.650852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:22.650852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:22.650852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:22.650852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:22.650852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:22.650852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:22.654692 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 375/759] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:22.655465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddb17b-9fb9-4b66-867b-92dd8ff8f00a None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:22.732109 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dcff6c-52c1-4ed5-9039-ae395eb85ab8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:22.759003 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddb17b-9fb9-4b66-867b-92dd8ff8f00a None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:22.769810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d57bf-791b-47f2-b54c-4abeb062cbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:22.770439 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d57bf-791b-47f2-b54c-4abeb062cbb7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:22.770723 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d57bf-791b-47f2-b54c-4abeb062cbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:22.770996 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d57bf-791b-47f2-b54c-4abeb062cbb7 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:22.771710 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d57bf-791b-47f2-b54c-4abeb062cbb7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:22.772017 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d57bf-791b-47f2-b54c-4abeb062cbb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:22.773933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d57bf-791b-47f2-b54c-4abeb062cbb7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:22.778762 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 385/760] 10.222.0.100 () {56 vars in 1129 bytes} [Tue Aug 2 08:58:22 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:22.779538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d57bf-791b-47f2-b54c-4abeb062cbb7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:22.798479 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff7c6c-0842-47ae-a753-ce7a40f26e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:22.799387 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff7c6c-0842-47ae-a753-ce7a40f26e6f None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:22.799674 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff7c6c-0842-47ae-a753-ce7a40f26e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:22.799946 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff7c6c-0842-47ae-a753-ce7a40f26e6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:22.810193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:22.813210 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff7c6c-0842-47ae-a753-ce7a40f26e6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:22.814827 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff7c6c-0842-47ae-a753-ce7a40f26e6f None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:22.830066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-71ff7c6c-0842-47ae-a753-ce7a40f26e6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:22.830066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:22.830066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:22.830066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:22.830066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:22.830066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:22.830066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:22.830066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:22.830066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:22.830066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:22.830066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:22.830066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:22.830066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:22.830066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:22.830066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:22.831172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:22.831172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:22.831172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:22.831172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:22.831172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:22.831172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:22.831172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:22.831172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:22.831172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:22.831172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:22.831172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:22.831172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:22.831172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:22.831172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:22.831172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:22.831172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:22.831172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:22.832097 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:22.832097 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:22.832097 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:22.832097 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:22.832097 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 376/761] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:22.832097 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff7c6c-0842-47ae-a753-ce7a40f26e6f None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:22.863978 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:22.892006 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d57bf-791b-47f2-b54c-4abeb062cbb7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:22.932721 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b55eea-ed2a-4a06-a467-9fa1ebd5841f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:22.933405 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b55eea-ed2a-4a06-a467-9fa1ebd5841f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:22.933690 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b55eea-ed2a-4a06-a467-9fa1ebd5841f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:22.933969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b55eea-ed2a-4a06-a467-9fa1ebd5841f None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:22.934663 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b55eea-ed2a-4a06-a467-9fa1ebd5841f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:22.934971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b55eea-ed2a-4a06-a467-9fa1ebd5841f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:22.936332 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b55eea-ed2a-4a06-a467-9fa1ebd5841f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:22.948889 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 386/762] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:22.949573 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b55eea-ed2a-4a06-a467-9fa1ebd5841f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:22.958627 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff7c6c-0842-47ae-a753-ce7a40f26e6f None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:22.994269 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8309d-c76d-4e41-8ead-6fe54f55097b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:22.994857 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8309d-c76d-4e41-8ead-6fe54f55097b None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:22.995140 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8309d-c76d-4e41-8ead-6fe54f55097b None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:22.995519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8309d-c76d-4e41-8ead-6fe54f55097b None admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:22.996263 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8309d-c76d-4e41-8ead-6fe54f55097b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:22.996689 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8309d-c76d-4e41-8ead-6fe54f55097b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:22.998205 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8309d-c76d-4e41-8ead-6fe54f55097b None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:23.017911 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 377/763] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:23.017911 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8309d-c76d-4e41-8ead-6fe54f55097b None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:23.037674 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:23.054826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b55eea-ed2a-4a06-a467-9fa1ebd5841f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:23.091878 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fb2bc-eee8-4924-b0a1-aed5624b848a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:23.092506 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fb2bc-eee8-4924-b0a1-aed5624b848a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:23.092809 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fb2bc-eee8-4924-b0a1-aed5624b848a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:23.093061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fb2bc-eee8-4924-b0a1-aed5624b848a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:23.093839 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fb2bc-eee8-4924-b0a1-aed5624b848a None admin] RBAC: Adding query filter params (interface=public, service_id=a7f59885831440409e7ede7c6cba0eb4, region_id=RegionOne) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:23.094151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fb2bc-eee8-4924-b0a1-aed5624b848a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:23.095503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fb2bc-eee8-4924-b0a1-aed5624b848a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:23.106055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 387/764] 10.222.0.100 () {56 vars in 1256 bytes} [Tue Aug 2 08:58:22 2022] GET /identity/v3/endpoints?service_id=a7f59885831440409e7ede7c6cba0eb4&interface=public®ion_id=RegionOne => generated 195 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:23.106873 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fb2bc-eee8-4924-b0a1-aed5624b848a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:23.119085 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:23.136746 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8309d-c76d-4e41-8ead-6fe54f55097b None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:23.174486 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ed3a4-dddd-490e-99e8-8795be117ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:23.175108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ed3a4-dddd-490e-99e8-8795be117ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:23.175395 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ed3a4-dddd-490e-99e8-8795be117ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:23.175672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ed3a4-dddd-490e-99e8-8795be117ed0 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:23.176366 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ed3a4-dddd-490e-99e8-8795be117ed0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:23.176671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ed3a4-dddd-490e-99e8-8795be117ed0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:23.178079 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ed3a4-dddd-490e-99e8-8795be117ed0 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:23.190326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 378/765] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:23 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:23.191140 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ed3a4-dddd-490e-99e8-8795be117ed0 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:23.218631 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fb2bc-eee8-4924-b0a1-aed5624b848a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:23.256028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6220a4-2235-4988-9a7f-90a642082422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:23.256679 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6220a4-2235-4988-9a7f-90a642082422 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:23.256964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6220a4-2235-4988-9a7f-90a642082422 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:23.257297 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6220a4-2235-4988-9a7f-90a642082422 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:23.263986 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6220a4-2235-4988-9a7f-90a642082422 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:23.265442 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6220a4-2235-4988-9a7f-90a642082422 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:23.276909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-1d6220a4-2235-4988-9a7f-90a642082422 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:23.276909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:23.276909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:23.276909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:23.276909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:23.276909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:23.276909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:23.276909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:23.276909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:23.276909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:23.276909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:23.276909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:23.276909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:23.276909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:23.276909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:23.277995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:23.277995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:23.277995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:23.277995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:23.277995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:23.277995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:23.277995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:23.277995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:23.277995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:23.277995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:23.277995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:23.277995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:23.277995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:23.277995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:23.277995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:23.277995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:23.277995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:23.279000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:23.279000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:23.279000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:23.279000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:23.279000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:23.279000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:23.279000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:23.279000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:23.279000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 388/766] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:23.279000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6220a4-2235-4988-9a7f-90a642082422 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:23.292086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:23.295756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ed3a4-dddd-490e-99e8-8795be117ed0 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:23.328695 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce207da6-20b3-4866-9266-87c8d6e29f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:23.329347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce207da6-20b3-4866-9266-87c8d6e29f88 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:23.329630 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce207da6-20b3-4866-9266-87c8d6e29f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:23.329909 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce207da6-20b3-4866-9266-87c8d6e29f88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:23.330688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce207da6-20b3-4866-9266-87c8d6e29f88 None admin] RBAC: Adding query filter params (role.id=7cbff51867a7475d837afd51f6c12f11, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=50cb532b58ed4951ba960a0a7d579b49) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:23.330999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce207da6-20b3-4866-9266-87c8d6e29f88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:23.332421 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce207da6-20b3-4866-9266-87c8d6e29f88 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:23.337897 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 379/767] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:58:23 2022] GET /identity/v3/role_assignments?user.id=50cb532b58ed4951ba960a0a7d579b49&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=7cbff51867a7475d837afd51f6c12f11 => generated 260 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:23.338668 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce207da6-20b3-4866-9266-87c8d6e29f88 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:23.351898 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:23.396984 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6220a4-2235-4988-9a7f-90a642082422 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:23.436063 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8999f2-f42e-475f-9d4e-a5fd1d90811f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:23.436810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8999f2-f42e-475f-9d4e-a5fd1d90811f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:23.437192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8999f2-f42e-475f-9d4e-a5fd1d90811f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:23.437546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8999f2-f42e-475f-9d4e-a5fd1d90811f None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:23.438290 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8999f2-f42e-475f-9d4e-a5fd1d90811f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:23.439748 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8999f2-f42e-475f-9d4e-a5fd1d90811f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:23.452667 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce207da6-20b3-4866-9266-87c8d6e29f88 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:23.453274 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 389/768] 10.222.0.100 () {56 vars in 1092 bytes} [Tue Aug 2 08:58:23 2022] GET /identity/v3/services => generated 776 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:23.454763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba227680-2ef5-4d37-90c1-31657b72239a None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:23.455142 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba227680-2ef5-4d37-90c1-31657b72239a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:23.455496 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba227680-2ef5-4d37-90c1-31657b72239a None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:23.457154 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 390/769] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:23.461882 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba227680-2ef5-4d37-90c1-31657b72239a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:23.508431 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b7cd8-afbb-400f-b31e-9591c762e62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:23.509000 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b7cd8-afbb-400f-b31e-9591c762e62b None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:23.509362 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b7cd8-afbb-400f-b31e-9591c762e62b None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:23.509660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b7cd8-afbb-400f-b31e-9591c762e62b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:23.518406 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b7cd8-afbb-400f-b31e-9591c762e62b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:23.520662 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b7cd8-afbb-400f-b31e-9591c762e62b None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:23.541228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-e27b7cd8-afbb-400f-b31e-9591c762e62b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:23.541228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:23.541228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:23.541228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:23.541228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:23.541228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:23.541228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:23.541228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:23.541228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:23.541228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:23.541228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:23.541228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:23.541228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:23.541228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:23.541228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:23.542414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:23.542414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:23.542414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:23.542414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:23.542414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:23.542414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:23.542414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:23.542414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:23.542414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:23.542414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:23.542414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:23.542414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:23.542414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:23.542414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:23.542414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:23.542414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:23.542414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:23.543537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:23.543537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:23.543537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:23.543537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:23.543537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:23.543537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:23.543537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:23.543537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:23.544431 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 380/770] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:23.547875 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0ab04-baf8-464d-b23d-852d86d75623 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:23.547875 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0ab04-baf8-464d-b23d-852d86d75623 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:23.547875 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0ab04-baf8-464d-b23d-852d86d75623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:23.567461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-60b0ab04-baf8-464d-b23d-852d86d75623 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:23.568777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-60b0ab04-baf8-464d-b23d-852d86d75623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:23.585945 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:23.622374 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba227680-2ef5-4d37-90c1-31657b72239a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:23.650660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:23.656614 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 381/771] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:23 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:23.657536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0ab04-baf8-464d-b23d-852d86d75623 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:23.677639 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6537882-046d-4e52-bc67-2370950400dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:23.678279 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6537882-046d-4e52-bc67-2370950400dd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:23.678566 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6537882-046d-4e52-bc67-2370950400dd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:23.678844 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6537882-046d-4e52-bc67-2370950400dd None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:23.679741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6537882-046d-4e52-bc67-2370950400dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:23.680056 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6537882-046d-4e52-bc67-2370950400dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:23.681524 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6537882-046d-4e52-bc67-2370950400dd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:23.695662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 391/772] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:23.696500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6537882-046d-4e52-bc67-2370950400dd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:23.777777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0ab04-baf8-464d-b23d-852d86d75623 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:23.801969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6537882-046d-4e52-bc67-2370950400dd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:23.814955 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c28fbc-bf79-4c95-b958-d370d829b34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:23.816003 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c28fbc-bf79-4c95-b958-d370d829b34e None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:23.816481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c28fbc-bf79-4c95-b958-d370d829b34e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:23.816951 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c28fbc-bf79-4c95-b958-d370d829b34e None admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:23.817915 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c28fbc-bf79-4c95-b958-d370d829b34e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:23.818437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c28fbc-bf79-4c95-b958-d370d829b34e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:23.820328 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c28fbc-bf79-4c95-b958-d370d829b34e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:23.838604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 382/773] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:23.838604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c28fbc-bf79-4c95-b958-d370d829b34e None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:23.845073 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2b8a6-24bf-4dc9-8f35-94adf75435e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:23.846093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2b8a6-24bf-4dc9-8f35-94adf75435e9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:23.846573 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2b8a6-24bf-4dc9-8f35-94adf75435e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:23.847039 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2b8a6-24bf-4dc9-8f35-94adf75435e9 None admin] PATH_INFO: `/v3/services/network` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:23.847914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf2b8a6-24bf-4dc9-8f35-94adf75435e9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:23.850036 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf2b8a6-24bf-4dc9-8f35-94adf75435e9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:23.862772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-fcf2b8a6-24bf-4dc9-8f35-94adf75435e9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 08:58:23.862772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:23.862772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:23.862772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:23.862772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:23.862772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:23.862772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:23.862772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:23.862772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 08:58:23.862772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:23.862772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:23.862772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:23.862772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:23.862772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:23.862772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:23.862772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:23.862772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:23.863832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:23.863832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:23.863832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:23.863832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:23.863832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:23.863832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:23.863832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:23.863832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:23.863832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:23.863832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:23.863832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:23.863832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:23.863832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:23.863832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:23.863832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:23.863832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:23.864682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:23.864682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:23.864682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:23.864682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:23.864682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:23.864682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:23.864682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:23.864682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:23.864682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:23.864682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:23.864682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:23.864682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:23.864682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 08:58:23.864682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:23.868642 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 392/774] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:23 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:23.873228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:23.874111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2b8a6-24bf-4dc9-8f35-94adf75435e9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:23.916482 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:23.964598 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c28fbc-bf79-4c95-b958-d370d829b34e None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:24.002879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591e5a8-4757-4b1e-be1e-c678cbadd139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:24.003706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591e5a8-4757-4b1e-be1e-c678cbadd139 None admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:24.004124 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591e5a8-4757-4b1e-be1e-c678cbadd139 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:24.004458 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591e5a8-4757-4b1e-be1e-c678cbadd139 None admin] PATH_INFO: `/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/49a0c1a4a6cf43e28ec1b8488c681848/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:24.006626 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e591e5a8-4757-4b1e-be1e-c678cbadd139 None admin] RBAC: Authorizing `identity:create_grant(project_id=61ca0176324e445b94e42a5214d2c89b, user_id=49a0c1a4a6cf43e28ec1b8488c681848, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:24.008587 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e591e5a8-4757-4b1e-be1e-c678cbadd139 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:24.014652 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2b8a6-24bf-4dc9-8f35-94adf75435e9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:24.029280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 383/775] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:58:23 2022] PUT /identity/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/49a0c1a4a6cf43e28ec1b8488c681848/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:58:24.030438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591e5a8-4757-4b1e-be1e-c678cbadd139 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:24.091045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1afa0-fb45-474b-a3b7-dfe5aa206c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:24.093760 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1afa0-fb45-474b-a3b7-dfe5aa206c3f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:24.094405 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1afa0-fb45-474b-a3b7-dfe5aa206c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:24.094911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1afa0-fb45-474b-a3b7-dfe5aa206c3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:24.097375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1afa0-fb45-474b-a3b7-dfe5aa206c3f None admin] RBAC: Adding query filter params (role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=1cd36b543bd9487ebd8ce578e9a76c78) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:24.098567 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1afa0-fb45-474b-a3b7-dfe5aa206c3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:24.102779 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1afa0-fb45-474b-a3b7-dfe5aa206c3f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:24.117364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 393/776] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:58:23 2022] GET /identity/v3/role_assignments?user.id=1cd36b543bd9487ebd8ce578e9a76c78&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 260 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:24.193893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591e5a8-4757-4b1e-be1e-c678cbadd139 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:24.218358 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:24.235444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe3b6e-b8cd-4879-b502-830ebbe41592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:24.236622 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe3b6e-b8cd-4879-b502-830ebbe41592 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:24.237343 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe3b6e-b8cd-4879-b502-830ebbe41592 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:24.237946 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe3b6e-b8cd-4879-b502-830ebbe41592 None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:24.239091 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe3b6e-b8cd-4879-b502-830ebbe41592 None admin] RBAC: Adding query filter params (name=network) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:24.239503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe3b6e-b8cd-4879-b502-830ebbe41592 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:24.241017 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe3b6e-b8cd-4879-b502-830ebbe41592 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:24.247350 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 384/777] 10.222.0.100 () {56 vars in 1117 bytes} [Tue Aug 2 08:58:24 2022] GET /identity/v3/services?name=network => generated 125 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:24.252869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1afa0-fb45-474b-a3b7-dfe5aa206c3f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:24.331099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:24.384489 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1afa0-fb45-474b-a3b7-dfe5aa206c3f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:24.420425 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5977d8-59bd-4b8a-922c-e0db19356617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:24.421249 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5977d8-59bd-4b8a-922c-e0db19356617 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:24.421632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5977d8-59bd-4b8a-922c-e0db19356617 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:24.422008 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5977d8-59bd-4b8a-922c-e0db19356617 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:24.422978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5977d8-59bd-4b8a-922c-e0db19356617 None admin] RBAC: Adding query filter params (type=network) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:24.423395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5977d8-59bd-4b8a-922c-e0db19356617 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:24.424835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5977d8-59bd-4b8a-922c-e0db19356617 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:24.430621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 394/778] 10.222.0.100 () {56 vars in 1117 bytes} [Tue Aug 2 08:58:24 2022] GET /identity/v3/services?type=network => generated 352 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:24.435949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe3b6e-b8cd-4879-b502-830ebbe41592 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:24.541987 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe3b6e-b8cd-4879-b502-830ebbe41592 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:24.578461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9ee17-501f-4516-9ae1-41bd52c03039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:24.579079 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9ee17-501f-4516-9ae1-41bd52c03039 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:24.579357 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9ee17-501f-4516-9ae1-41bd52c03039 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:24.579747 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9ee17-501f-4516-9ae1-41bd52c03039 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:24.580503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9ee17-501f-4516-9ae1-41bd52c03039 None admin] RBAC: Adding query filter params (interface=public, service_id=a6d9bbd810154fdebe86f6aff1971b75, region_id=RegionOne) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:24.580810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9ee17-501f-4516-9ae1-41bd52c03039 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:24.582253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9ee17-501f-4516-9ae1-41bd52c03039 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:24.591026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 385/779] 10.222.0.100 () {56 vars in 1256 bytes} [Tue Aug 2 08:58:24 2022] GET /identity/v3/endpoints?service_id=a6d9bbd810154fdebe86f6aff1971b75&interface=public®ion_id=RegionOne => generated 195 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:24.597319 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5977d8-59bd-4b8a-922c-e0db19356617 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:24.722671 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5977d8-59bd-4b8a-922c-e0db19356617 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:24.749969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:24.760729 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b611ae-033a-4c19-ac49-a0570eba1254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:24.761511 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b611ae-033a-4c19-ac49-a0570eba1254 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:24.761856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b611ae-033a-4c19-ac49-a0570eba1254 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:24.762248 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b611ae-033a-4c19-ac49-a0570eba1254 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:24.762970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b611ae-033a-4c19-ac49-a0570eba1254 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:24.764375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b611ae-033a-4c19-ac49-a0570eba1254 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:24.770223 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 395/780] 10.222.0.100 () {56 vars in 1092 bytes} [Tue Aug 2 08:58:24 2022] GET /identity/v3/services => generated 776 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:25.805553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e67300-ead4-490b-9708-5b9bde68fe1c None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:25.806069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e67300-ead4-490b-9708-5b9bde68fe1c None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:25.806180 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e67300-ead4-490b-9708-5b9bde68fe1c None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:25.807711 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 386/781] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:25 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:25.809714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de12bb-d679-4e04-8ab0-e741e11a02a2 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:25.810095 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de12bb-d679-4e04-8ab0-e741e11a02a2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:25.810256 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de12bb-d679-4e04-8ab0-e741e11a02a2 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:25.811829 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 396/782] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:25.816596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1562607-799b-4fbb-b9aa-005a167cdedd None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:25.816882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1562607-799b-4fbb-b9aa-005a167cdedd None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:25.817184 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1562607-799b-4fbb-b9aa-005a167cdedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:25.823402 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c11dac-1e6e-4742-a27e-75a7e74df84b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:25.823688 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c11dac-1e6e-4742-a27e-75a7e74df84b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:25.823966 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c11dac-1e6e-4742-a27e-75a7e74df84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:25.842840 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-f1562607-799b-4fbb-b9aa-005a167cdedd None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:25.844028 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f1562607-799b-4fbb-b9aa-005a167cdedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:25.858334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-46c11dac-1e6e-4742-a27e-75a7e74df84b None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:25.859788 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-46c11dac-1e6e-4742-a27e-75a7e74df84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:25.942383 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:25.962247 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 397/783] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:25 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:58:25.983417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 387/784] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:25 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:25.984470 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c11dac-1e6e-4742-a27e-75a7e74df84b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:26.018967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1562607-799b-4fbb-b9aa-005a167cdedd None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:26.140858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1562607-799b-4fbb-b9aa-005a167cdedd None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:26.181824 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d8afe-f13a-4f25-a32b-99dda35c6540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:26.182480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d8afe-f13a-4f25-a32b-99dda35c6540 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:26.182757 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d8afe-f13a-4f25-a32b-99dda35c6540 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:26.183029 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d8afe-f13a-4f25-a32b-99dda35c6540 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:26.191210 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c11dac-1e6e-4742-a27e-75a7e74df84b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:26.195618 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d8afe-f13a-4f25-a32b-99dda35c6540 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:26.201807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-c80d8afe-f13a-4f25-a32b-99dda35c6540 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:58:26.201807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:26.201807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:26.201807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:26.201807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:26.201807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:26.201807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:26.201807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:26.201807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:26.201807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:26.201807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:26.201807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:26.201807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:26.201807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:26.201807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:26.201807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:26.202777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:26.202777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:26.202777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:26.202777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:26.202777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:26.202777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:26.202777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:26.202777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:26.202777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:26.202777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:26.202777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:26.202777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:26.202777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:26.202777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:26.202777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:26.202777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:26.203604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:26.203604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:26.203604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:26.203604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:26.203604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:26.203604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:26.203604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:26.203604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:26.203604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:26.203604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:26.203604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:58:26.203604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:26.203604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 388/785] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:58:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:26.213485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d8afe-f13a-4f25-a32b-99dda35c6540 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:26.234708 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:26.264747 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e39b7-6461-4939-9310-3f5fdd219e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:26.266404 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e39b7-6461-4939-9310-3f5fdd219e66 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:26.266991 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e39b7-6461-4939-9310-3f5fdd219e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:26.268503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e39b7-6461-4939-9310-3f5fdd219e66 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:26.269332 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e39b7-6461-4939-9310-3f5fdd219e66 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:26.271538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e39b7-6461-4939-9310-3f5fdd219e66 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:26.286140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-0a5e39b7-6461-4939-9310-3f5fdd219e66 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 08:58:26.286140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:26.286140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:26.286140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:26.286140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:26.286140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:26.286140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:26.286140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:26.286140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 08:58:26.286140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:26.286140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:26.286140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:26.286140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:26.286140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:26.286140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:26.286140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:26.287287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:26.287287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:26.287287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:26.287287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:26.287287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:26.287287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:26.287287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:26.287287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:26.287287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:26.287287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:26.287287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:26.287287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:26.287287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:26.287287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:26.287287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:26.287287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:26.288964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:26.288964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:26.288964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:26.288964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:26.288964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:26.288964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:26.288964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:26.288964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:26.288964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:26.288964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:26.288964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:26.288964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:26.288964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:26.288964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 08:58:26.288964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:26.296558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:26.296759 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 398/786] 10.222.0.100 () {56 vars in 1134 bytes} [Tue Aug 2 08:58:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:26.302647 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e39b7-6461-4939-9310-3f5fdd219e66 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:26.376529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d8afe-f13a-4f25-a32b-99dda35c6540 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:26.425354 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc94ea-25fe-4962-8fb9-900c0855974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:26.426236 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc94ea-25fe-4962-8fb9-900c0855974a None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:26.426665 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc94ea-25fe-4962-8fb9-900c0855974a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:26.427085 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc94ea-25fe-4962-8fb9-900c0855974a None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:26.427965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc94ea-25fe-4962-8fb9-900c0855974a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:26.428429 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc94ea-25fe-4962-8fb9-900c0855974a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:26.430029 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc94ea-25fe-4962-8fb9-900c0855974a None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:26.454094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 389/787] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:58:26 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:26.461135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc94ea-25fe-4962-8fb9-900c0855974a None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:26.537378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e39b7-6461-4939-9310-3f5fdd219e66 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:26.593297 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714e983-47a6-49f6-a2f1-7a36bd6d6ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:26.594599 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714e983-47a6-49f6-a2f1-7a36bd6d6ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:26.594895 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714e983-47a6-49f6-a2f1-7a36bd6d6ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:26.595186 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714e983-47a6-49f6-a2f1-7a36bd6d6ef1 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:26.595879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714e983-47a6-49f6-a2f1-7a36bd6d6ef1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:26.596191 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714e983-47a6-49f6-a2f1-7a36bd6d6ef1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:26.597608 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714e983-47a6-49f6-a2f1-7a36bd6d6ef1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:26.603020 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 399/788] 10.222.0.100 () {56 vars in 1129 bytes} [Tue Aug 2 08:58:26 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:26.604210 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452dbb0f-cf93-4c56-9147-32bac5388dd5 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:26.604497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452dbb0f-cf93-4c56-9147-32bac5388dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:26.604774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452dbb0f-cf93-4c56-9147-32bac5388dd5 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:26.610780 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 400/789] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:26 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:26.611689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452dbb0f-cf93-4c56-9147-32bac5388dd5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:26.618839 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:26.623519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc94ea-25fe-4962-8fb9-900c0855974a None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:26.660734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a2bdd-e134-40ff-a1c7-cdb9d96a0915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:26.661748 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a2bdd-e134-40ff-a1c7-cdb9d96a0915 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:26.662208 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a2bdd-e134-40ff-a1c7-cdb9d96a0915 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:26.662660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a2bdd-e134-40ff-a1c7-cdb9d96a0915 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:26.670492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:26.674835 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a2bdd-e134-40ff-a1c7-cdb9d96a0915 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:26.676547 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a2bdd-e134-40ff-a1c7-cdb9d96a0915 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:26.691517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-a18a2bdd-e134-40ff-a1c7-cdb9d96a0915 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:26.691517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:26.691517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:26.691517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:26.691517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:26.691517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:26.691517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:26.691517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:26.691517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:26.691517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:26.691517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:26.691517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:26.691517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:26.691517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:26.691517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:26.692427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:26.692427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:26.692427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:26.692427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:26.692427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:26.692427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:26.692427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:26.692427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:26.692427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:26.692427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:26.692427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:26.692427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:26.692427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:26.692427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:26.692427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:26.692427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:26.692427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:26.693396 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:26.693396 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:26.693396 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:26.693396 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:26.695702 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 390/790] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:26.701459 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f9602-1a27-4ecd-9fce-d141c3d5cfc4 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:26.701965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f9602-1a27-4ecd-9fce-d141c3d5cfc4 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:26.702423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f9602-1a27-4ecd-9fce-d141c3d5cfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:26.721762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-569f9602-1a27-4ecd-9fce-d141c3d5cfc4 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:26.723207 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-569f9602-1a27-4ecd-9fce-d141c3d5cfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:26.727848 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452dbb0f-cf93-4c56-9147-32bac5388dd5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:26.764812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0808d-c023-4ec5-bebc-5b75159590b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:26.765686 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0808d-c023-4ec5-bebc-5b75159590b6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:26.765971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0808d-c023-4ec5-bebc-5b75159590b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:26.766251 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0808d-c023-4ec5-bebc-5b75159590b6 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:26.766933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0808d-c023-4ec5-bebc-5b75159590b6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:26.767293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0808d-c023-4ec5-bebc-5b75159590b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:26.768655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0808d-c023-4ec5-bebc-5b75159590b6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:26.779672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 401/791] 10.222.0.100 () {56 vars in 1129 bytes} [Tue Aug 2 08:58:26 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:26.780458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0808d-c023-4ec5-bebc-5b75159590b6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:26.796788 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 391/792] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:26 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:26.798487 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d73c1-c574-49e3-bae1-cd9ed16861d5 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:26.798851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d73c1-c574-49e3-bae1-cd9ed16861d5 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:26.799204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d73c1-c574-49e3-bae1-cd9ed16861d5 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:26.800682 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 392/793] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:26.805077 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d73c1-c574-49e3-bae1-cd9ed16861d5 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:26.896507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0808d-c023-4ec5-bebc-5b75159590b6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:26.898849 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:26.912961 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:26.922272 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d73c1-c574-49e3-bae1-cd9ed16861d5 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:26.938625 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4659315-f784-4669-9c51-4fa8a33974d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:26.939180 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4659315-f784-4669-9c51-4fa8a33974d7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:26.939466 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4659315-f784-4669-9c51-4fa8a33974d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:26.939767 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4659315-f784-4669-9c51-4fa8a33974d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:26.940442 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4659315-f784-4669-9c51-4fa8a33974d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:26.940966 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4659315-f784-4669-9c51-4fa8a33974d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:26.942133 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4659315-f784-4669-9c51-4fa8a33974d7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:26.955159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 402/794] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:26.956430 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea1f81-747a-40d6-be20-1a5642b76f1d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:26.956723 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea1f81-747a-40d6-be20-1a5642b76f1d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:26.957002 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea1f81-747a-40d6-be20-1a5642b76f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:26.963792 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87eddc6-bec3-4e6b-9dbe-74a1dd36171b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:26.964549 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87eddc6-bec3-4e6b-9dbe-74a1dd36171b None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:26.964904 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87eddc6-bec3-4e6b-9dbe-74a1dd36171b None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:26.965218 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87eddc6-bec3-4e6b-9dbe-74a1dd36171b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:26.965929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87eddc6-bec3-4e6b-9dbe-74a1dd36171b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:26.967290 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87eddc6-bec3-4e6b-9dbe-74a1dd36171b None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:26.975002 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f2ea1f81-747a-40d6-be20-1a5642b76f1d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:26.977003 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f2ea1f81-747a-40d6-be20-1a5642b76f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:26.991002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 393/795] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 08:58:26 2022] POST /identity/v3/endpoints => generated 348 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:58:26.992306 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87eddc6-bec3-4e6b-9dbe-74a1dd36171b None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:27.094879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 403/796] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:26 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:27.097279 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea1f81-747a-40d6-be20-1a5642b76f1d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:27.154734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87eddc6-bec3-4e6b-9dbe-74a1dd36171b None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:27.196969 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4894f-254e-47c2-b05e-3efa5e3f4aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:27.197917 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4894f-254e-47c2-b05e-3efa5e3f4aef None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:27.198204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4894f-254e-47c2-b05e-3efa5e3f4aef None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:27.198486 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4894f-254e-47c2-b05e-3efa5e3f4aef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:27.208219 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:27.215148 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4894f-254e-47c2-b05e-3efa5e3f4aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:27.222291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-f4f4894f-254e-47c2-b05e-3efa5e3f4aef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:58:27.222291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:27.222291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:27.222291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:27.222291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:27.222291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:27.222291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:27.222291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:27.222291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:27.222291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:27.222291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:27.222291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:27.222291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:27.222291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:27.222291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:27.222291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:27.223718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:27.223718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:27.223718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:27.223718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:27.223718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:27.223718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:27.223718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:27.223718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:27.223718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:27.223718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:27.223718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:27.223718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:27.223718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:27.223718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:27.223718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:27.223718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:27.224584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:27.224584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:27.224584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:27.224584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:27.224584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:27.224584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:27.224584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:27.224584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:27.224584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:27.224584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:27.224584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:58:27.224584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:27.224584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 394/797] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:58:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:27.224584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4894f-254e-47c2-b05e-3efa5e3f4aef None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:27.235748 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:27.247001 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea1f81-747a-40d6-be20-1a5642b76f1d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:27.283601 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d769b-b407-44b9-8444-8ac58cc0721d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:27.284251 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d769b-b407-44b9-8444-8ac58cc0721d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:27.284548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d769b-b407-44b9-8444-8ac58cc0721d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:27.284818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d769b-b407-44b9-8444-8ac58cc0721d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:27.292802 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d769b-b407-44b9-8444-8ac58cc0721d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:27.294269 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d769b-b407-44b9-8444-8ac58cc0721d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:27.306160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-bd9d769b-b407-44b9-8444-8ac58cc0721d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:58:27.306160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:27.306160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:27.306160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:27.306160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:27.306160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:27.306160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:27.306160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:27.306160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:27.306160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:27.306160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:27.306160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:27.306160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:27.306160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:27.306160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:27.306160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:27.307138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:27.307138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:27.307138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:27.307138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:27.307138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:27.307138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:27.307138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:27.307138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:27.307138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:27.307138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:27.307138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:27.307138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:27.307138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:27.307138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:27.307138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:27.307138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:27.307138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:27.307138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:27.308112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:27.308112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:27.308112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:27.308112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:27.308112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:27.308112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:27.308112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:27.308112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:27.308112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:58:27.308112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:27.308112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 404/798] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:58:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:27.308112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d769b-b407-44b9-8444-8ac58cc0721d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:27.344349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4894f-254e-47c2-b05e-3efa5e3f4aef None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:27.380013 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52902caf-5780-4220-90fd-6b6e509facbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:27.380654 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52902caf-5780-4220-90fd-6b6e509facbe None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:27.380936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52902caf-5780-4220-90fd-6b6e509facbe None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:27.381250 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52902caf-5780-4220-90fd-6b6e509facbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:27.392033 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52902caf-5780-4220-90fd-6b6e509facbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:27.396971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-52902caf-5780-4220-90fd-6b6e509facbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:27.396971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:27.396971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:27.396971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:27.396971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:27.396971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:27.396971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:27.396971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:27.396971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:27.396971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:27.396971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:27.396971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:27.396971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:27.396971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:27.396971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:27.396971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:27.398999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:27.398999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:27.398999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:27.398999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:27.398999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:27.398999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:27.398999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:27.398999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:27.398999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:27.398999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:27.398999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:27.398999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:27.398999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:27.398999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:27.398999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:27.398999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:27.400193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:27.400193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:27.400193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:27.400193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:27.400193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:27.400193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:27.400193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:27.400193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:27.400193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:27.400193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:27.400193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:27.400193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:27.400193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 395/799] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:58:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:27.404674 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ff755-f967-4f03-80bf-f98e7b23ef83 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:27.404965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ff755-f967-4f03-80bf-f98e7b23ef83 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:27.405277 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ff755-f967-4f03-80bf-f98e7b23ef83 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:27.406560 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 396/800] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:27 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:27.407259 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ff755-f967-4f03-80bf-f98e7b23ef83 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:27.425279 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d769b-b407-44b9-8444-8ac58cc0721d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:27.463310 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c3130-0284-43c9-a650-1c1a1a2a7963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:27.463854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c3130-0284-43c9-a650-1c1a1a2a7963 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:27.464141 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c3130-0284-43c9-a650-1c1a1a2a7963 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:27.464418 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c3130-0284-43c9-a650-1c1a1a2a7963 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:27.465160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c3130-0284-43c9-a650-1c1a1a2a7963 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:27.465470 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c3130-0284-43c9-a650-1c1a1a2a7963 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:27.466829 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c3130-0284-43c9-a650-1c1a1a2a7963 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:27.472713 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:27.475479 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 405/801] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:58:27 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:27.480774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c3130-0284-43c9-a650-1c1a1a2a7963 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:27.499786 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:27.530639 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ff755-f967-4f03-80bf-f98e7b23ef83 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:27.567667 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97135402-1f29-4dea-b120-cf6f9f7c9c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:27.568283 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97135402-1f29-4dea-b120-cf6f9f7c9c4a None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:27.568561 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97135402-1f29-4dea-b120-cf6f9f7c9c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:27.568928 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97135402-1f29-4dea-b120-cf6f9f7c9c4a None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:27.569786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97135402-1f29-4dea-b120-cf6f9f7c9c4a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:27.570186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97135402-1f29-4dea-b120-cf6f9f7c9c4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:27.571601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97135402-1f29-4dea-b120-cf6f9f7c9c4a None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:27.586109 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c3130-0284-43c9-a650-1c1a1a2a7963 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:27.591282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 397/802] 10.222.0.100 () {56 vars in 1140 bytes} [Tue Aug 2 08:58:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:27.596725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c198cb9-371a-48bc-9f47-40cc76e3de35 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:27.597018 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c198cb9-371a-48bc-9f47-40cc76e3de35 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:27.597474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c198cb9-371a-48bc-9f47-40cc76e3de35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:27.615997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-6c198cb9-371a-48bc-9f47-40cc76e3de35 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:27.617731 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-6c198cb9-371a-48bc-9f47-40cc76e3de35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:27.623728 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b9703-3928-4a14-be64-506233cab014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:27.624357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b9703-3928-4a14-be64-506233cab014 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:27.624641 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b9703-3928-4a14-be64-506233cab014 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:27.624922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b9703-3928-4a14-be64-506233cab014 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:27.625937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b9703-3928-4a14-be64-506233cab014 None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:27.626251 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b9703-3928-4a14-be64-506233cab014 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:27.627606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b9703-3928-4a14-be64-506233cab014 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:27.641039 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 406/803] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 08:58:27 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:27.641039 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b9703-3928-4a14-be64-506233cab014 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:27.694086 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 398/804] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:27 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:27.698375 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c198cb9-371a-48bc-9f47-40cc76e3de35 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:27.726740 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:27.747421 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b9703-3928-4a14-be64-506233cab014 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:27.760577 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:27.785314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2491ac-1d0b-4d64-b7cb-fe77b6a5d820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:27.785966 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2491ac-1d0b-4d64-b7cb-fe77b6a5d820 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:27.786251 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2491ac-1d0b-4d64-b7cb-fe77b6a5d820 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:27.786528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2491ac-1d0b-4d64-b7cb-fe77b6a5d820 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:27.797878 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2491ac-1d0b-4d64-b7cb-fe77b6a5d820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:27.799374 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2491ac-1d0b-4d64-b7cb-fe77b6a5d820 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:27.813525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ab2491ac-1d0b-4d64-b7cb-fe77b6a5d820 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:27.813525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:27.813525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:27.813525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:27.813525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:27.813525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:27.813525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:27.813525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:27.813525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:27.813525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:27.813525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:27.813525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:27.813525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:27.813525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:27.813525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:27.814465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:27.814465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:27.814465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:27.814465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:27.814465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:27.814465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:27.814465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:27.814465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:27.814465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:27.814465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:27.814465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:27.814465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:27.814465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:27.814465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:27.814465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:27.814465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:27.814465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:27.815376 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:27.815376 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:27.815376 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:27.815376 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:27.815376 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 407/805] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:27.815651 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2491ac-1d0b-4d64-b7cb-fe77b6a5d820 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:27.826493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c198cb9-371a-48bc-9f47-40cc76e3de35 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:27.862149 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db3617-d5e1-4f90-9984-4d2e63744368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:27.862778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db3617-d5e1-4f90-9984-4d2e63744368 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:27.863055 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db3617-d5e1-4f90-9984-4d2e63744368 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:27.863325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db3617-d5e1-4f90-9984-4d2e63744368 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:27.873952 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db3617-d5e1-4f90-9984-4d2e63744368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:27.875651 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db3617-d5e1-4f90-9984-4d2e63744368 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:27.890553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-08db3617-d5e1-4f90-9984-4d2e63744368 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:27.890553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:27.890553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:27.890553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:27.890553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:27.890553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:27.890553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:27.890553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:27.890553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:27.890553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:27.890553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:27.890553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:27.890553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:27.890553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:27.890553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:27.891424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:27.891424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:27.891424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:27.891424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:27.891424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:27.891424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:27.891424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:27.891424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:27.891424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:27.891424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:27.891424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:27.891424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:27.891424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:27.891424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:27.891424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:27.891424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:27.891424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:27.892312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:27.892312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:27.892312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:27.892312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:27.892312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 399/806] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:27.892592 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db3617-d5e1-4f90-9984-4d2e63744368 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:27.936887 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2491ac-1d0b-4d64-b7cb-fe77b6a5d820 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:27.971214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773e699-f0f8-476a-a8d5-2f7e9baf556c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:27.971926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773e699-f0f8-476a-a8d5-2f7e9baf556c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:27.972200 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773e699-f0f8-476a-a8d5-2f7e9baf556c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:27.972474 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773e699-f0f8-476a-a8d5-2f7e9baf556c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:27.975216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:27.985288 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773e699-f0f8-476a-a8d5-2f7e9baf556c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:27.986661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773e699-f0f8-476a-a8d5-2f7e9baf556c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:28.001773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-7773e699-f0f8-476a-a8d5-2f7e9baf556c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:28.001773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:28.001773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:28.001773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:28.001773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:28.001773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:28.001773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:28.001773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:28.001773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:28.001773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:28.001773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:28.001773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:28.001773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:28.001773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:28.001773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:28.002659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:28.002659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:28.002659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:28.002659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:28.002659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:28.002659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:28.002659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:28.002659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:28.002659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:28.002659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:28.002659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:28.002659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:28.002659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:28.002659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:28.002659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:28.002659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:28.002659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:28.003575 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:28.003575 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:28.003575 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:28.003575 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:28.003575 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 408/807] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:28.003871 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773e699-f0f8-476a-a8d5-2f7e9baf556c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:28.010128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:28.013861 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db3617-d5e1-4f90-9984-4d2e63744368 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:28.059521 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58bfab-26a9-485c-8cc9-9294449397e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:28.060899 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58bfab-26a9-485c-8cc9-9294449397e5 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:28.061415 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58bfab-26a9-485c-8cc9-9294449397e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:28.061772 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58bfab-26a9-485c-8cc9-9294449397e5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:28.062482 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58bfab-26a9-485c-8cc9-9294449397e5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:28.063857 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58bfab-26a9-485c-8cc9-9294449397e5 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:28.073624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-da58bfab-26a9-485c-8cc9-9294449397e5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 08:58:28.073624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:28.073624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:28.073624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:28.073624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:28.073624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:28.073624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:28.073624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:28.073624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 08:58:28.073624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:28.073624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:28.073624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:28.073624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:28.073624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:28.073624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:28.073624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:28.073624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:28.074596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:28.074596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:28.074596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:28.074596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:28.074596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:28.074596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:28.074596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:28.074596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:28.074596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:28.074596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:28.074596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:28.074596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:28.074596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:28.074596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:28.074596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:28.074596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:28.075444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:28.075444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:28.075444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:28.075444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:28.075444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:28.075444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:28.075444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:28.075444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:28.075444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:28.075444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:28.075444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:28.075444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:28.075444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 08:58:28.075444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:28.075444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 400/808] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:27 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:28.076252 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58bfab-26a9-485c-8cc9-9294449397e5 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:28.114884 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773e699-f0f8-476a-a8d5-2f7e9baf556c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:28.150962 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c901baa-0132-41a5-899e-b88e0e241d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:28.151598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c901baa-0132-41a5-899e-b88e0e241d6c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:28.151883 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c901baa-0132-41a5-899e-b88e0e241d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:28.152168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c901baa-0132-41a5-899e-b88e0e241d6c None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:28.152866 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c901baa-0132-41a5-899e-b88e0e241d6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:28.153208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c901baa-0132-41a5-899e-b88e0e241d6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:28.154669 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c901baa-0132-41a5-899e-b88e0e241d6c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:28.168010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 409/809] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:28.169080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c901baa-0132-41a5-899e-b88e0e241d6c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:28.195568 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58bfab-26a9-485c-8cc9-9294449397e5 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:28.222007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:28.232991 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e83455-95a2-43ed-af23-2eac96eb9798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:28.233660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e83455-95a2-43ed-af23-2eac96eb9798 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:28.233946 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e83455-95a2-43ed-af23-2eac96eb9798 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:28.234227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e83455-95a2-43ed-af23-2eac96eb9798 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:28.234923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e83455-95a2-43ed-af23-2eac96eb9798 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:28.235235 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e83455-95a2-43ed-af23-2eac96eb9798 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:28.236674 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e83455-95a2-43ed-af23-2eac96eb9798 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:28.252536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 401/810] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:28.252536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e83455-95a2-43ed-af23-2eac96eb9798 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:28.275204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:28.286124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c901baa-0132-41a5-899e-b88e0e241d6c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:28.324187 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d401b-333a-44ed-a73d-9a97177e5114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:28.325100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d401b-333a-44ed-a73d-9a97177e5114 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:28.325426 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d401b-333a-44ed-a73d-9a97177e5114 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:28.325702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d401b-333a-44ed-a73d-9a97177e5114 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:28.326397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d401b-333a-44ed-a73d-9a97177e5114 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:28.326707 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d401b-333a-44ed-a73d-9a97177e5114 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:28.328066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d401b-333a-44ed-a73d-9a97177e5114 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:28.341261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 410/811] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:28.346390 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d401b-333a-44ed-a73d-9a97177e5114 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:28.360460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e83455-95a2-43ed-af23-2eac96eb9798 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:28.396594 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badfb25d-6ff5-4026-b7fc-50304007f599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:28.397255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfb25d-6ff5-4026-b7fc-50304007f599 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:28.398117 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfb25d-6ff5-4026-b7fc-50304007f599 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:28.398394 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfb25d-6ff5-4026-b7fc-50304007f599 None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:28.399070 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badfb25d-6ff5-4026-b7fc-50304007f599 None admin] RBAC: Adding query filter params (name=network) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:28.399378 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badfb25d-6ff5-4026-b7fc-50304007f599 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:28.400748 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badfb25d-6ff5-4026-b7fc-50304007f599 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:28.405426 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 402/812] 10.222.0.100 () {56 vars in 1117 bytes} [Tue Aug 2 08:58:28 2022] GET /identity/v3/services?name=network => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:28.405957 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badfb25d-6ff5-4026-b7fc-50304007f599 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:28.461283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d401b-333a-44ed-a73d-9a97177e5114 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:28.468349 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:28.497176 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:28.499094 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b0759-a9ba-45b0-af65-66e1ab66cfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:28.499800 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b0759-a9ba-45b0-af65-66e1ab66cfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:28.500363 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b0759-a9ba-45b0-af65-66e1ab66cfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:28.500620 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b0759-a9ba-45b0-af65-66e1ab66cfb1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:28.509274 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b0759-a9ba-45b0-af65-66e1ab66cfb1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:28.510855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b0759-a9ba-45b0-af65-66e1ab66cfb1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:28.514301 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badfb25d-6ff5-4026-b7fc-50304007f599 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:28.523611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-847b0759-a9ba-45b0-af65-66e1ab66cfb1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:58:28.523611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:28.523611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:28.523611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:28.523611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:28.523611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:28.523611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:28.523611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:28.523611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:28.523611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:28.523611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:28.523611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:28.523611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:28.523611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:28.523611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:28.523611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:28.524597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:28.524597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:28.524597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:28.524597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:28.524597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:28.524597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:28.524597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:28.524597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:28.524597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:28.524597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:28.524597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:28.524597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:28.524597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:28.524597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:28.524597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:28.524597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:28.524597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:28.524597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:28.525588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:28.525588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:28.525588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:28.525588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:28.525588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:28.525588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:28.525588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:28.525588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:28.525588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:58:28.525588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:28.525588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 411/813] 10.222.0.100 () {56 vars in 1098 bytes} [Tue Aug 2 08:58:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:28.525588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b0759-a9ba-45b0-af65-66e1ab66cfb1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:28.555758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62056500-5008-486c-8df5-8060ced397c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:28.556731 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62056500-5008-486c-8df5-8060ced397c3 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:28.557017 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62056500-5008-486c-8df5-8060ced397c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:28.557492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62056500-5008-486c-8df5-8060ced397c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:28.566412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62056500-5008-486c-8df5-8060ced397c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:28.568851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62056500-5008-486c-8df5-8060ced397c3 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:28.582186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-62056500-5008-486c-8df5-8060ced397c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:28.582186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:28.582186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:28.582186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:28.582186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:28.582186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:28.582186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:28.582186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:28.582186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:28.582186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:28.582186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:28.582186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:28.582186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:28.582186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:28.582186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:28.583109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:28.583109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:28.583109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:28.583109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:28.583109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:28.583109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:28.583109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:28.583109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:28.583109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:28.583109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:28.583109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:28.583109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:28.583109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:28.583109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:28.583109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:28.583109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:28.583109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:28.584009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:28.584009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:28.584009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:28.584009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:28.584009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:28.584009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:28.584009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:28.584009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:28.584009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 403/814] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:28.585977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62056500-5008-486c-8df5-8060ced397c3 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:28.642927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b0759-a9ba-45b0-af65-66e1ab66cfb1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:28.677426 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df428ab1-8274-4b95-8d38-22afaea7adce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:28.678077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df428ab1-8274-4b95-8d38-22afaea7adce None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:28.678369 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df428ab1-8274-4b95-8d38-22afaea7adce None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:28.679366 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df428ab1-8274-4b95-8d38-22afaea7adce None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:28.687504 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df428ab1-8274-4b95-8d38-22afaea7adce None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:28.688945 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df428ab1-8274-4b95-8d38-22afaea7adce None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:28.691233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62056500-5008-486c-8df5-8060ced397c3 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:28.701179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-df428ab1-8274-4b95-8d38-22afaea7adce None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:58:28.701179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:28.701179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:28.701179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:28.701179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:28.701179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:28.701179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:28.701179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:28.701179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:28.701179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:28.701179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:28.701179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:28.701179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:28.701179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:28.701179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:28.701179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:28.702159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:28.702159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:28.702159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:28.702159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:28.702159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:28.702159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:28.702159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:28.702159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:28.702159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:28.702159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:28.702159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:28.702159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:28.702159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:28.702159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:28.702159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:28.702159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:28.702159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:28.702159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:28.703117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:28.703117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:28.703117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:28.703117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:28.703117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:28.703117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:28.703117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:28.703117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:28.703117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:58:28.703117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:28.703117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 412/815] 10.222.0.100 () {56 vars in 1122 bytes} [Tue Aug 2 08:58:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:28.703117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df428ab1-8274-4b95-8d38-22afaea7adce None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:28.733290 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d7578-77c3-4b79-96b9-e21dee4c34f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:28.734049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:28.735192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d7578-77c3-4b79-96b9-e21dee4c34f9 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:28.735684 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d7578-77c3-4b79-96b9-e21dee4c34f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:28.736287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d7578-77c3-4b79-96b9-e21dee4c34f9 None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:28.737174 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d7578-77c3-4b79-96b9-e21dee4c34f9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:28.737684 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d7578-77c3-4b79-96b9-e21dee4c34f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:28.739993 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d7578-77c3-4b79-96b9-e21dee4c34f9 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:28.751504 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 404/816] 10.222.0.100 () {56 vars in 1117 bytes} [Tue Aug 2 08:58:28 2022] GET /identity/v3/services?type=network => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:28.752576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d7578-77c3-4b79-96b9-e21dee4c34f9 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:28.766918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:28.811061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df428ab1-8274-4b95-8d38-22afaea7adce None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:28.847288 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b75e3-4485-4aaa-a9f1-f02b76bd09fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:28.848343 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b75e3-4485-4aaa-a9f1-f02b76bd09fd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:28.848807 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b75e3-4485-4aaa-a9f1-f02b76bd09fd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:28.849295 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b75e3-4485-4aaa-a9f1-f02b76bd09fd None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:28.850401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b75e3-4485-4aaa-a9f1-f02b76bd09fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:28.850943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b75e3-4485-4aaa-a9f1-f02b76bd09fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:28.852782 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b75e3-4485-4aaa-a9f1-f02b76bd09fd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:28.859991 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d7578-77c3-4b79-96b9-e21dee4c34f9 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:28.872723 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 413/817] 10.222.0.100 () {56 vars in 1138 bytes} [Tue Aug 2 08:58:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:28.873932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b75e3-4485-4aaa-a9f1-f02b76bd09fd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:28.900338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551ccbc-16f9-44ed-b045-ebcf78fd0165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:28.901288 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551ccbc-16f9-44ed-b045-ebcf78fd0165 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:28.901714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551ccbc-16f9-44ed-b045-ebcf78fd0165 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:28.902147 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551ccbc-16f9-44ed-b045-ebcf78fd0165 None admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:28.902920 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551ccbc-16f9-44ed-b045-ebcf78fd0165 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:28.903208 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551ccbc-16f9-44ed-b045-ebcf78fd0165 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:28.906649 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551ccbc-16f9-44ed-b045-ebcf78fd0165 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:28.922859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 405/818] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:28.923750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551ccbc-16f9-44ed-b045-ebcf78fd0165 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:28.972969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:28.991784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b75e3-4485-4aaa-a9f1-f02b76bd09fd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:28.995546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:29.027080 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551ccbc-16f9-44ed-b045-ebcf78fd0165 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:29.031008 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bb25d-1c65-4c2b-864e-20912c9df4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:29.031634 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bb25d-1c65-4c2b-864e-20912c9df4ec None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:29.031916 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bb25d-1c65-4c2b-864e-20912c9df4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:29.032187 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bb25d-1c65-4c2b-864e-20912c9df4ec None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:29.032979 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bb25d-1c65-4c2b-864e-20912c9df4ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:29.033319 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bb25d-1c65-4c2b-864e-20912c9df4ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:29.034851 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bb25d-1c65-4c2b-864e-20912c9df4ec None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:29.053733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 414/819] 10.222.0.100 () {56 vars in 1154 bytes} [Tue Aug 2 08:58:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:29.054564 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bb25d-1c65-4c2b-864e-20912c9df4ec None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:29.068592 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427d954-dd0a-4abf-9c90-049f65d0ad3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:29.069292 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427d954-dd0a-4abf-9c90-049f65d0ad3e None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:29.069580 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427d954-dd0a-4abf-9c90-049f65d0ad3e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:29.069857 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427d954-dd0a-4abf-9c90-049f65d0ad3e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:29.070572 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427d954-dd0a-4abf-9c90-049f65d0ad3e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:29.071968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427d954-dd0a-4abf-9c90-049f65d0ad3e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:29.092665 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 406/820] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 08:58:28 2022] POST /identity/v3/endpoints => generated 339 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:58:29.093493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427d954-dd0a-4abf-9c90-049f65d0ad3e None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:29.171611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bb25d-1c65-4c2b-864e-20912c9df4ec None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:29.210961 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427d954-dd0a-4abf-9c90-049f65d0ad3e None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:29.214118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb5815-3c69-4699-978c-0bb0a5b64b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:29.214866 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb5815-3c69-4699-978c-0bb0a5b64b42 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:29.215153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb5815-3c69-4699-978c-0bb0a5b64b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:29.215432 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb5815-3c69-4699-978c-0bb0a5b64b42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:29.230120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:29.236949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:29.254554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4b862-f630-4181-a99b-802f329d2953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:29.255279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4b862-f630-4181-a99b-802f329d2953 None admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:29.255621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4b862-f630-4181-a99b-802f329d2953 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:29.255921 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4b862-f630-4181-a99b-802f329d2953 None admin] PATH_INFO: `/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/50cb532b58ed4951ba960a0a7d579b49/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:29.258533 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e4b862-f630-4181-a99b-802f329d2953 None admin] RBAC: Authorizing `identity:create_grant(project_id=61ca0176324e445b94e42a5214d2c89b, user_id=50cb532b58ed4951ba960a0a7d579b49, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:29.260137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e4b862-f630-4181-a99b-802f329d2953 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:29.279162 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 407/821] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:58:29 2022] PUT /identity/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/50cb532b58ed4951ba960a0a7d579b49/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:58:29.285412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4b862-f630-4181-a99b-802f329d2953 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:29.301587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb5815-3c69-4699-978c-0bb0a5b64b42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:29.303057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb5815-3c69-4699-978c-0bb0a5b64b42 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:29.411610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4b862-f630-4181-a99b-802f329d2953 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:29.428028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 415/822] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:58:29 2022] GET /identity/v3/auth/tokens => generated 1163 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:58:29.429943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b189555-9625-442e-96eb-0c4de2180d5a None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:29.430230 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b189555-9625-442e-96eb-0c4de2180d5a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:29.430502 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b189555-9625-442e-96eb-0c4de2180d5a None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:29.432028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 416/823] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:29.434669 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b189555-9625-442e-96eb-0c4de2180d5a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:29.457504 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610c4be-459d-4bfe-bfc0-7060338acd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:29.458145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610c4be-459d-4bfe-bfc0-7060338acd5b None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:29.458426 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610c4be-459d-4bfe-bfc0-7060338acd5b None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:29.458697 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610c4be-459d-4bfe-bfc0-7060338acd5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:29.471017 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610c4be-459d-4bfe-bfc0-7060338acd5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:29.472534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610c4be-459d-4bfe-bfc0-7060338acd5b None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:29.484760 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:29.490820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-5610c4be-459d-4bfe-bfc0-7060338acd5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:29.490820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:29.490820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:29.490820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:29.490820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:29.490820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:29.490820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:29.490820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:29.490820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:29.490820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:29.490820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:29.490820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:29.490820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:29.490820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:29.490820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:29.492409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:29.492409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:29.492409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:29.492409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:29.492409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:29.492409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:29.492409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:29.492409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:29.492409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:29.492409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:29.492409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:29.492409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:29.492409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:29.492409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:29.492409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:29.492409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:29.492409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:29.493343 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:29.493343 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:29.493343 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:29.493343 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:29.493343 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 408/824] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:29.494145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3aae2-0052-41eb-bea5-4bde9237a876 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:29.494417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3aae2-0052-41eb-bea5-4bde9237a876 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:29.494686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3aae2-0052-41eb-bea5-4bde9237a876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:29.513361 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-50f3aae2-0052-41eb-bea5-4bde9237a876 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:29.514664 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-50f3aae2-0052-41eb-bea5-4bde9237a876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:29.542575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:29.552790 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b189555-9625-442e-96eb-0c4de2180d5a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:29.588860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5bdb0-2cac-42b2-93b4-0254720b1a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:29.589861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5bdb0-2cac-42b2-93b4-0254720b1a26 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:29.590337 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5bdb0-2cac-42b2-93b4-0254720b1a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:29.590799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5bdb0-2cac-42b2-93b4-0254720b1a26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:29.595244 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 409/825] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:29 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:29.596272 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f3aae2-0052-41eb-bea5-4bde9237a876 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:29.610105 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e5bdb0-2cac-42b2-93b4-0254720b1a26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:29.612115 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e5bdb0-2cac-42b2-93b4-0254720b1a26 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:29.631127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-22e5bdb0-2cac-42b2-93b4-0254720b1a26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:29.631127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:29.631127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:29.631127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:29.631127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:29.631127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:29.631127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:29.631127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:29.631127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:29.631127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:29.631127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:29.631127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:29.631127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:29.631127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:29.631127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:29.632254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:29.632254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:29.632254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:29.632254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:29.632254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:29.632254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:29.632254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:29.632254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:29.632254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:29.632254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:29.632254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:29.632254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:29.632254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:29.632254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:29.632254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:29.632254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:29.632254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:29.633329 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:29.633329 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:29.633329 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:29.633329 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:29.633329 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 417/826] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:29.634056 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5bdb0-2cac-42b2-93b4-0254720b1a26 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:29.725571 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f3aae2-0052-41eb-bea5-4bde9237a876 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:29.730810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:29.739676 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5bdb0-2cac-42b2-93b4-0254720b1a26 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:29.762514 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db47e7-d154-4b3b-94b2-a8bd250ac048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:29.763121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db47e7-d154-4b3b-94b2-a8bd250ac048 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:29.763404 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db47e7-d154-4b3b-94b2-a8bd250ac048 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:29.763679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db47e7-d154-4b3b-94b2-a8bd250ac048 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:29.764370 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db47e7-d154-4b3b-94b2-a8bd250ac048 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:29.764671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db47e7-d154-4b3b-94b2-a8bd250ac048 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:29.766087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db47e7-d154-4b3b-94b2-a8bd250ac048 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:29.775682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9748e-0452-4db0-9060-eca720cd1e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:29.776741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9748e-0452-4db0-9060-eca720cd1e09 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:29.777097 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9748e-0452-4db0-9060-eca720cd1e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:29.777424 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9748e-0452-4db0-9060-eca720cd1e09 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:29.782668 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 410/827] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:29.782790 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9748e-0452-4db0-9060-eca720cd1e09 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:29.782870 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db47e7-d154-4b3b-94b2-a8bd250ac048 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:29.782963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9748e-0452-4db0-9060-eca720cd1e09 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:29.791412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-f0d9748e-0452-4db0-9060-eca720cd1e09 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 08:58:29.791412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:29.791412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:29.791412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:29.791412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:29.791412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:29.791412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:29.791412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:29.791412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 08:58:29.791412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:29.791412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:29.791412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:29.791412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:29.791412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:29.791412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:29.791412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:29.791412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:29.792427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:29.792427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:29.792427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:29.792427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:29.792427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:29.792427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:29.792427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:29.792427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:29.792427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:29.792427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:29.792427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:29.792427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:29.792427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:29.792427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:29.792427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:29.792427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:29.793341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:29.793341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:29.793341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:29.793341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:29.793341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:29.793341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:29.793341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:29.793341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:29.793341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:29.793341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:29.793341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:29.793341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:29.793341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 08:58:29.793341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:29.793341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 418/828] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:58:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:29.794154 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9748e-0452-4db0-9060-eca720cd1e09 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:29.814155 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:29.895851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db47e7-d154-4b3b-94b2-a8bd250ac048 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:29.898450 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9748e-0452-4db0-9060-eca720cd1e09 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:29.937249 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67b2f0-8ab1-4c29-9e08-74c38a44d10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:29.937659 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67b2f0-8ab1-4c29-9e08-74c38a44d10d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:29.937934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67b2f0-8ab1-4c29-9e08-74c38a44d10d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:29.938226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67b2f0-8ab1-4c29-9e08-74c38a44d10d None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:29.938926 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67b2f0-8ab1-4c29-9e08-74c38a44d10d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:29.939177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67b2f0-8ab1-4c29-9e08-74c38a44d10d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:29.940527 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67b2f0-8ab1-4c29-9e08-74c38a44d10d None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:29.942260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9a682-0f46-4867-b195-13fca1d664c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:29.942908 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9a682-0f46-4867-b195-13fca1d664c9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:29.943194 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9a682-0f46-4867-b195-13fca1d664c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:29.943475 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9a682-0f46-4867-b195-13fca1d664c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:29.951220 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9a682-0f46-4867-b195-13fca1d664c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:29.952642 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9a682-0f46-4867-b195-13fca1d664c9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:29.958392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 411/829] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:29.958392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67b2f0-8ab1-4c29-9e08-74c38a44d10d None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:29.971387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-d0f9a682-0f46-4867-b195-13fca1d664c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:29.971387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:29.971387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:29.971387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:29.971387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:29.971387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:29.971387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:29.971387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:29.971387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:29.971387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:29.971387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:29.971387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:29.971387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:29.971387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:29.971387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:29.972333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:29.972333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:29.972333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:29.972333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:29.972333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:29.972333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:29.972333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:29.972333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:29.972333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:29.972333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:29.972333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:29.972333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:29.972333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:29.972333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:29.972333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:29.972333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:29.972333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:29.979388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:29.979388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:29.979388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:29.979388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:29.979388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:29.979388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:29.979388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:29.979388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:29.979388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 419/830] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:29.979388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9a682-0f46-4867-b195-13fca1d664c9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:30.001630 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:30.046756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:30.064155 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67b2f0-8ab1-4c29-9e08-74c38a44d10d None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:30.090672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9a682-0f46-4867-b195-13fca1d664c9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:30.102048 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c12cf-c7be-4ce8-b78b-14b8af0d40de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:30.102707 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c12cf-c7be-4ce8-b78b-14b8af0d40de None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:30.102954 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c12cf-c7be-4ce8-b78b-14b8af0d40de None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:30.103234 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c12cf-c7be-4ce8-b78b-14b8af0d40de None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:30.103907 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c12cf-c7be-4ce8-b78b-14b8af0d40de None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:30.104225 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c12cf-c7be-4ce8-b78b-14b8af0d40de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:30.105630 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c12cf-c7be-4ce8-b78b-14b8af0d40de None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:30.110401 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 412/831] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:58:29 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:30.115826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c12cf-c7be-4ce8-b78b-14b8af0d40de None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:30.126136 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e668fb-7b1a-442e-8d78-6341fb52ce07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:30.126785 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e668fb-7b1a-442e-8d78-6341fb52ce07 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:30.127075 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e668fb-7b1a-442e-8d78-6341fb52ce07 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:30.127359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e668fb-7b1a-442e-8d78-6341fb52ce07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:30.134466 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e668fb-7b1a-442e-8d78-6341fb52ce07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:30.135907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e668fb-7b1a-442e-8d78-6341fb52ce07 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:30.147801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-16e668fb-7b1a-442e-8d78-6341fb52ce07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:30.147801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:30.147801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:30.147801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:30.147801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:30.147801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:30.147801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:30.147801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:30.147801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:30.147801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:30.147801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:30.147801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:30.147801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:30.147801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:30.147801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:30.148727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:30.148727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:30.148727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:30.148727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:30.148727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:30.148727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:30.148727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:30.148727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:30.148727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:30.148727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:30.148727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:30.148727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:30.148727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:30.148727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:30.148727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:30.148727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:30.148727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:30.149705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:30.149705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:30.149705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:30.149705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:30.149705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:30.149705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:30.149705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:30.149705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:30.153733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 420/832] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:30.154441 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e668fb-7b1a-442e-8d78-6341fb52ce07 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:30.227039 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c12cf-c7be-4ce8-b78b-14b8af0d40de None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:30.236382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:30.260205 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e668fb-7b1a-442e-8d78-6341fb52ce07 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:30.264851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31314dc1-3acc-42c5-b948-9bd7442b981d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:30.265586 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31314dc1-3acc-42c5-b948-9bd7442b981d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:30.265874 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31314dc1-3acc-42c5-b948-9bd7442b981d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:30.266152 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31314dc1-3acc-42c5-b948-9bd7442b981d None admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:30.266899 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31314dc1-3acc-42c5-b948-9bd7442b981d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:30.267211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31314dc1-3acc-42c5-b948-9bd7442b981d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:30.268683 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31314dc1-3acc-42c5-b948-9bd7442b981d None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:30.286071 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 413/833] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:30.286071 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31314dc1-3acc-42c5-b948-9bd7442b981d None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:30.295100 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:30.303389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bb915-6073-445a-b35c-c88d7913ec04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:30.304020 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bb915-6073-445a-b35c-c88d7913ec04 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:30.304306 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bb915-6073-445a-b35c-c88d7913ec04 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:30.304582 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bb915-6073-445a-b35c-c88d7913ec04 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:30.305285 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bb915-6073-445a-b35c-c88d7913ec04 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:30.305591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bb915-6073-445a-b35c-c88d7913ec04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:30.306931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bb915-6073-445a-b35c-c88d7913ec04 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:30.311745 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 421/834] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:58:30 2022] GET /identity/v3/services?type=volumev3 => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:30.318572 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bb915-6073-445a-b35c-c88d7913ec04 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:30.420390 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31314dc1-3acc-42c5-b948-9bd7442b981d None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:30.455229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bb915-6073-445a-b35c-c88d7913ec04 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:30.472956 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3a0fa-bb9c-4bc7-bdf8-fedae87f87a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:30.474514 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3a0fa-bb9c-4bc7-bdf8-fedae87f87a9 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:30.474514 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3a0fa-bb9c-4bc7-bdf8-fedae87f87a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:30.474514 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3a0fa-bb9c-4bc7-bdf8-fedae87f87a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:30.474923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3a0fa-bb9c-4bc7-bdf8-fedae87f87a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:30.475230 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3a0fa-bb9c-4bc7-bdf8-fedae87f87a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:30.477983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3a0fa-bb9c-4bc7-bdf8-fedae87f87a9 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:30.492065 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 414/835] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:30.497732 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3a0fa-bb9c-4bc7-bdf8-fedae87f87a9 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:30.510545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022996a-0aae-4f24-a93c-7389234f6f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:30.511225 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022996a-0aae-4f24-a93c-7389234f6f42 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:30.511499 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022996a-0aae-4f24-a93c-7389234f6f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:30.511783 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022996a-0aae-4f24-a93c-7389234f6f42 None admin] PATH_INFO: `/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/1cd36b543bd9487ebd8ce578e9a76c78/roles/6e6ac8e8e7044b209d1a8b5bb6e4b1d8` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:30.530202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:30.532563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022996a-0aae-4f24-a93c-7389234f6f42 None admin] RBAC: Authorizing `identity:create_grant(project_id=61ca0176324e445b94e42a5214d2c89b, user_id=1cd36b543bd9487ebd8ce578e9a76c78, role_id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:30.534197 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022996a-0aae-4f24-a93c-7389234f6f42 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:30.557750 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 422/836] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:58:30 2022] PUT /identity/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/1cd36b543bd9487ebd8ce578e9a76c78/roles/6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:58:30.585770 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:30.626868 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3a0fa-bb9c-4bc7-bdf8-fedae87f87a9 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:30.676508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb9fda-d05e-48b4-a6d3-523b6d26054c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:30.677344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb9fda-d05e-48b4-a6d3-523b6d26054c None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:30.677651 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb9fda-d05e-48b4-a6d3-523b6d26054c None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:30.677925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb9fda-d05e-48b4-a6d3-523b6d26054c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:30.679337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb9fda-d05e-48b4-a6d3-523b6d26054c None admin] RBAC: Adding query filter params (role.id=7cbff51867a7475d837afd51f6c12f11, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=49a0c1a4a6cf43e28ec1b8488c681848) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:30.679654 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb9fda-d05e-48b4-a6d3-523b6d26054c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:30.681134 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb9fda-d05e-48b4-a6d3-523b6d26054c None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:30.688429 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 415/837] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:58:30 2022] GET /identity/v3/role_assignments?user.id=49a0c1a4a6cf43e28ec1b8488c681848&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=7cbff51867a7475d837afd51f6c12f11 => generated 610 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:31.373198 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582eb3be-e699-42c8-b047-be2f3819df8d None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:31.373716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582eb3be-e699-42c8-b047-be2f3819df8d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:31.373825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582eb3be-e699-42c8-b047-be2f3819df8d None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:31.375668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 423/838] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:31.381059 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c551b6-e554-4f14-a9d7-c608b028c87f None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:31.382096 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c551b6-e554-4f14-a9d7-c608b028c87f None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:31.382568 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c551b6-e554-4f14-a9d7-c608b028c87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:31.403204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-10c551b6-e554-4f14-a9d7-c608b028c87f None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:31.404419 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-10c551b6-e554-4f14-a9d7-c608b028c87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:31.487949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 416/839] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:31 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:31.514064 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582eb3be-e699-42c8-b047-be2f3819df8d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:31.639163 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582eb3be-e699-42c8-b047-be2f3819df8d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:31.674779 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85081793-9a58-4cf8-8d1e-19377b6f7939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:31.675468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85081793-9a58-4cf8-8d1e-19377b6f7939 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:31.675760 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85081793-9a58-4cf8-8d1e-19377b6f7939 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:31.676034 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85081793-9a58-4cf8-8d1e-19377b6f7939 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:31.686839 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85081793-9a58-4cf8-8d1e-19377b6f7939 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:31.692224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-85081793-9a58-4cf8-8d1e-19377b6f7939 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:58:31.692224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:31.692224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:31.692224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:31.692224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:31.692224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:31.692224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:31.692224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:31.692224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:31.692224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:31.692224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:31.692224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:31.692224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:31.692224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:31.692224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:31.692224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:31.693156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:31.693156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:31.693156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:31.693156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:31.693156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:31.693156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:31.693156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:31.693156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:31.693156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:31.693156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:31.693156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:31.693156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:31.693156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:31.693156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:31.693156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:31.693156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:31.694589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:31.694589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:31.694589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:31.694589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:31.694589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:31.694589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:31.694589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:31.694589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:31.694589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:31.694589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:31.694589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:58:31.694589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:31.694589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 424/840] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:58:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:31.700133 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c551b6-e554-4f14-a9d7-c608b028c87f None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:31.764079 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:31.805520 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c551b6-e554-4f14-a9d7-c608b028c87f None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:31.840677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963537b-0a82-44cc-8918-cd852539f8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:31.841326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963537b-0a82-44cc-8918-cd852539f8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:31.841743 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963537b-0a82-44cc-8918-cd852539f8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:31.842018 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963537b-0a82-44cc-8918-cd852539f8f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:31.842730 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963537b-0a82-44cc-8918-cd852539f8f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:31.843030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963537b-0a82-44cc-8918-cd852539f8f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:31.844393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963537b-0a82-44cc-8918-cd852539f8f3 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:31.853502 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 417/841] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:58:31 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:31.860917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85081793-9a58-4cf8-8d1e-19377b6f7939 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:31.925053 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:31.980054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85081793-9a58-4cf8-8d1e-19377b6f7939 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:32.015718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71de8dd-21fd-43fe-8e68-611fd6462d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:32.016373 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71de8dd-21fd-43fe-8e68-611fd6462d3e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:32.016655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71de8dd-21fd-43fe-8e68-611fd6462d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:32.016928 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71de8dd-21fd-43fe-8e68-611fd6462d3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:32.028349 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71de8dd-21fd-43fe-8e68-611fd6462d3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:32.029864 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71de8dd-21fd-43fe-8e68-611fd6462d3e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:32.045509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-f71de8dd-21fd-43fe-8e68-611fd6462d3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:32.045509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:32.045509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:32.045509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:32.045509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:32.045509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:32.045509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:32.045509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:32.045509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:32.045509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:32.045509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:32.045509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:32.045509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:32.045509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:32.045509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:32.046531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:32.046531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:32.046531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:32.046531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:32.046531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:32.046531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:32.046531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:32.046531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:32.046531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:32.046531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:32.046531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:32.046531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:32.046531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:32.046531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:32.046531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:32.046531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:32.046531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:32.047445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:32.047445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:32.047445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:32.047445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:32.047445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 425/842] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:32.054763 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963537b-0a82-44cc-8918-cd852539f8f3 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:32.164010 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963537b-0a82-44cc-8918-cd852539f8f3 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:32.193609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:32.198191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f80854-8c23-48a8-839f-c644a2d77d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:32.198812 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f80854-8c23-48a8-839f-c644a2d77d09 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:32.199081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f80854-8c23-48a8-839f-c644a2d77d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:32.199362 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f80854-8c23-48a8-839f-c644a2d77d09 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:32.200241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f80854-8c23-48a8-839f-c644a2d77d09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:32.200733 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f80854-8c23-48a8-839f-c644a2d77d09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:32.202241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f80854-8c23-48a8-839f-c644a2d77d09 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:32.215164 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 418/843] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:32.222193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71de8dd-21fd-43fe-8e68-611fd6462d3e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:32.340961 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71de8dd-21fd-43fe-8e68-611fd6462d3e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:32.343402 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:32.376415 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d1fdf-4681-4a17-aa33-e09730d75437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:32.377310 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d1fdf-4681-4a17-aa33-e09730d75437 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:32.377475 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d1fdf-4681-4a17-aa33-e09730d75437 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:32.377626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d1fdf-4681-4a17-aa33-e09730d75437 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:32.386017 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d1fdf-4681-4a17-aa33-e09730d75437 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:32.387452 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d1fdf-4681-4a17-aa33-e09730d75437 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:32.399433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ba6d1fdf-4681-4a17-aa33-e09730d75437 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:58:32.399433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:32.399433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:32.399433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:32.399433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:32.399433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:32.399433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:32.399433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:32.399433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:32.399433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:32.399433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:32.399433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:32.399433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:32.399433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:32.399433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:32.399433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:32.401632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:32.401632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:32.401632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:32.401632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:32.401632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:32.401632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:32.401632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:32.401632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:32.401632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:32.401632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:32.401632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:32.401632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:32.401632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:32.401632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:32.401632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:32.401632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:32.401632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:32.401632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:32.403854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:32.403854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:32.403854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:32.403854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:32.403854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:32.403854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:32.403854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:32.403854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:32.403854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:58:32.403854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:32.403854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 426/844] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:58:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:32.410194 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f80854-8c23-48a8-839f-c644a2d77d09 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:32.420407 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb39451-df0b-4aca-9a69-65a00b03aa49 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:32.420948 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb39451-df0b-4aca-9a69-65a00b03aa49 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:32.421430 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb39451-df0b-4aca-9a69-65a00b03aa49 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:32.423302 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 427/845] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:32 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:58:32.427920 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886ca57-535d-4a6c-a77d-d7f2d6430bb3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:32.428209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886ca57-535d-4a6c-a77d-d7f2d6430bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:32.428606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886ca57-535d-4a6c-a77d-d7f2d6430bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:32.451542 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d886ca57-535d-4a6c-a77d-d7f2d6430bb3 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:32.453286 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d886ca57-535d-4a6c-a77d-d7f2d6430bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:32.533011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f80854-8c23-48a8-839f-c644a2d77d09 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:32.539463 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 428/846] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:32 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:58:32.561884 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886ca57-535d-4a6c-a77d-d7f2d6430bb3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:32.572414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18baeb3d-347e-4a4d-b74b-27aa0b5c54b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:32.573143 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18baeb3d-347e-4a4d-b74b-27aa0b5c54b9 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:32.573558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18baeb3d-347e-4a4d-b74b-27aa0b5c54b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:32.573910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18baeb3d-347e-4a4d-b74b-27aa0b5c54b9 None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:32.574838 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18baeb3d-347e-4a4d-b74b-27aa0b5c54b9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:32.575223 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18baeb3d-347e-4a4d-b74b-27aa0b5c54b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:32.576845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18baeb3d-347e-4a4d-b74b-27aa0b5c54b9 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:32.601198 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 419/847] 10.222.0.100 () {56 vars in 1140 bytes} [Tue Aug 2 08:58:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:32.605088 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18baeb3d-347e-4a4d-b74b-27aa0b5c54b9 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:32.659112 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:32.688712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:32.698997 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886ca57-535d-4a6c-a77d-d7f2d6430bb3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:32.729064 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18baeb3d-347e-4a4d-b74b-27aa0b5c54b9 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:32.747182 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd1f5e-d577-4402-9c05-d670e9a240d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:32.748069 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd1f5e-d577-4402-9c05-d670e9a240d9 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:32.748362 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd1f5e-d577-4402-9c05-d670e9a240d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:32.748641 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd1f5e-d577-4402-9c05-d670e9a240d9 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:32.749421 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd1f5e-d577-4402-9c05-d670e9a240d9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:32.751478 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd1f5e-d577-4402-9c05-d670e9a240d9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:32.769069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146d6bf-a430-4bf2-abcc-89e01f36faad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:32.770194 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146d6bf-a430-4bf2-abcc-89e01f36faad None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:32.770661 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146d6bf-a430-4bf2-abcc-89e01f36faad None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:32.771117 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146d6bf-a430-4bf2-abcc-89e01f36faad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:32.781215 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 429/848] 10.222.0.100 () {60 vars in 1146 bytes} [Tue Aug 2 08:58:32 2022] POST /identity/v3/services => generated 252 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 08:58:32.861072 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146d6bf-a430-4bf2-abcc-89e01f36faad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:32.863018 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146d6bf-a430-4bf2-abcc-89e01f36faad None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:32.868957 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d25c8b-4fd0-4aed-99e7-8a26b0867814 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:32.869462 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d25c8b-4fd0-4aed-99e7-8a26b0867814 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:32.869845 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d25c8b-4fd0-4aed-99e7-8a26b0867814 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:32.871748 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 430/849] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:32 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:58:32.877675 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f03406-d2c2-4a17-a77a-174d9aa3db67 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:32.878078 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f03406-d2c2-4a17-a77a-174d9aa3db67 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:32.878457 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f03406-d2c2-4a17-a77a-174d9aa3db67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:32.899933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-43f03406-d2c2-4a17-a77a-174d9aa3db67 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:32.905613 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-43f03406-d2c2-4a17-a77a-174d9aa3db67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:32.919111 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:32.989496 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 420/850] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:58:32 2022] GET /identity/v3/auth/tokens => generated 1163 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:58:32.991476 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479c889-8d25-4290-8efa-6ff2931df615 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:32.991874 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479c889-8d25-4290-8efa-6ff2931df615 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:32.992294 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479c889-8d25-4290-8efa-6ff2931df615 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:32.993838 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 421/851] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:32 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:32.995971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479c889-8d25-4290-8efa-6ff2931df615 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:32.998993 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 431/852] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:32 2022] POST /identity/v3/auth/tokens => generated 1264 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:33.000526 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2577e-272e-4194-a5d4-fca01242ac00 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:33.000815 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2577e-272e-4194-a5d4-fca01242ac00 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:33.001089 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2577e-272e-4194-a5d4-fca01242ac00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:33.029695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8fb2577e-272e-4194-a5d4-fca01242ac00 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:33.031505 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8fb2577e-272e-4194-a5d4-fca01242ac00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:33.104779 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 432/853] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:32 2022] POST /identity/v3/auth/tokens => generated 1264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:33.105926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2577e-272e-4194-a5d4-fca01242ac00 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:33.117683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:33.118012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479c889-8d25-4290-8efa-6ff2931df615 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:33.158971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831de5f8-37e9-4a0d-9bd9-16eac0573513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:33.159921 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831de5f8-37e9-4a0d-9bd9-16eac0573513 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:33.160265 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831de5f8-37e9-4a0d-9bd9-16eac0573513 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:33.160604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831de5f8-37e9-4a0d-9bd9-16eac0573513 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:33.171692 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831de5f8-37e9-4a0d-9bd9-16eac0573513 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:33.173244 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831de5f8-37e9-4a0d-9bd9-16eac0573513 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:33.175227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:33.191466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-831de5f8-37e9-4a0d-9bd9-16eac0573513 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:33.191466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:33.191466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:33.191466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:33.191466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:33.191466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:33.191466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:33.191466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:33.191466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:33.191466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:33.191466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:33.191466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:33.191466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:33.191466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:33.191466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:33.192596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:33.192596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:33.192596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:33.192596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:33.192596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:33.192596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:33.192596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:33.192596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:33.192596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:33.192596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:33.192596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:33.192596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:33.192596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:33.192596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:33.192596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:33.192596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:33.192596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:33.193718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:33.193718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:33.193718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:33.193718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:33.199099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 422/854] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:33.200001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831de5f8-37e9-4a0d-9bd9-16eac0573513 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:33.216308 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2577e-272e-4194-a5d4-fca01242ac00 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:33.258957 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa412a4-6557-41c6-8aba-7444a3fb0ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:33.259617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa412a4-6557-41c6-8aba-7444a3fb0ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:33.259897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa412a4-6557-41c6-8aba-7444a3fb0ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:33.260183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa412a4-6557-41c6-8aba-7444a3fb0ba5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:33.260823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa412a4-6557-41c6-8aba-7444a3fb0ba5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:33.262222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa412a4-6557-41c6-8aba-7444a3fb0ba5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:33.271428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-2fa412a4-6557-41c6-8aba-7444a3fb0ba5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 08:58:33.271428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:33.271428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:33.271428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:33.271428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:33.271428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:33.271428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:33.271428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:33.271428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 08:58:33.271428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:33.271428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:33.271428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:33.271428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:33.271428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:33.271428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:33.271428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:33.273368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:33.273368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:33.273368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:33.273368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:33.273368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:33.273368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:33.273368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:33.273368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:33.273368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:33.273368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:33.273368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:33.273368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:33.273368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:33.273368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:33.273368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:33.273368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:33.274325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:33.274325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:33.274325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:33.274325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:33.274325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:33.274325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:33.274325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:33.274325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:33.274325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:33.274325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:33.274325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:33.274325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:33.274325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:33.274325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 08:58:33.274325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:33.274325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 433/855] 10.222.0.100 () {56 vars in 1137 bytes} [Tue Aug 2 08:58:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:33.275185 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa412a4-6557-41c6-8aba-7444a3fb0ba5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:33.320017 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831de5f8-37e9-4a0d-9bd9-16eac0573513 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:33.356696 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06da15e-42fe-4b15-b745-88f4c29f2be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:33.357472 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06da15e-42fe-4b15-b745-88f4c29f2be8 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:33.357872 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06da15e-42fe-4b15-b745-88f4c29f2be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:33.358219 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06da15e-42fe-4b15-b745-88f4c29f2be8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:33.360480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:33.369505 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06da15e-42fe-4b15-b745-88f4c29f2be8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:33.375428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-a06da15e-42fe-4b15-b745-88f4c29f2be8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:33.375428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:33.375428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:33.375428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:33.375428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:33.375428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:33.375428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:33.375428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:33.375428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:33.375428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:33.375428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:33.375428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:33.375428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:33.375428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:33.375428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:33.375428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:33.376534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:33.376534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:33.376534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:33.376534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:33.376534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:33.376534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:33.376534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:33.376534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:33.376534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:33.376534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:33.376534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:33.376534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:33.376534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:33.376534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:33.376534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:33.376534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:33.377583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:33.377583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:33.377583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:33.377583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:33.377583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:33.377583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:33.377583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:33.377583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:33.377583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:33.377583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:33.377583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:58:33.377583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:33.378804 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 423/856] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 08:58:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:33.379482 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06da15e-42fe-4b15-b745-88f4c29f2be8 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:33.394309 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa412a4-6557-41c6-8aba-7444a3fb0ba5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:33.429316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fb3eb-3345-4ad0-b62d-4f98feb90b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:33.429939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fb3eb-3345-4ad0-b62d-4f98feb90b79 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:33.430222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fb3eb-3345-4ad0-b62d-4f98feb90b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:33.430500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fb3eb-3345-4ad0-b62d-4f98feb90b79 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:33.431187 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fb3eb-3345-4ad0-b62d-4f98feb90b79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:33.431494 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fb3eb-3345-4ad0-b62d-4f98feb90b79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:33.432848 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fb3eb-3345-4ad0-b62d-4f98feb90b79 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:33.444646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 434/857] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:33.445491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fb3eb-3345-4ad0-b62d-4f98feb90b79 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:33.451338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:33.501778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06da15e-42fe-4b15-b745-88f4c29f2be8 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:33.538809 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a3277-580d-495e-8b49-8e3cf34d2e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:33.539347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a3277-580d-495e-8b49-8e3cf34d2e78 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:33.539640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a3277-580d-495e-8b49-8e3cf34d2e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:33.539911 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a3277-580d-495e-8b49-8e3cf34d2e78 None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:33.540593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a3277-580d-495e-8b49-8e3cf34d2e78 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:33.541018 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a3277-580d-495e-8b49-8e3cf34d2e78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:33.542532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a3277-580d-495e-8b49-8e3cf34d2e78 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:33.548097 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 424/858] 10.222.0.100 () {56 vars in 1131 bytes} [Tue Aug 2 08:58:33 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:33.548953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a3277-580d-495e-8b49-8e3cf34d2e78 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:33.555414 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fb3eb-3345-4ad0-b62d-4f98feb90b79 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:33.592541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6075a-d859-45b7-9a37-f0bcb0c658a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:33.593189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6075a-d859-45b7-9a37-f0bcb0c658a3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:33.593473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6075a-d859-45b7-9a37-f0bcb0c658a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:33.593747 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6075a-d859-45b7-9a37-f0bcb0c658a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:33.594455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e6075a-d859-45b7-9a37-f0bcb0c658a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:33.594760 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e6075a-d859-45b7-9a37-f0bcb0c658a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:33.596110 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e6075a-d859-45b7-9a37-f0bcb0c658a3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:33.597512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:33.605565 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 435/859] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 08:58:33 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:33.606359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6075a-d859-45b7-9a37-f0bcb0c658a3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:33.661990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a3277-580d-495e-8b49-8e3cf34d2e78 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:33.680183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:33.700123 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532b6f7-f4a8-4c38-bd5f-1ee2102790d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:33.700967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532b6f7-f4a8-4c38-bd5f-1ee2102790d7 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:33.701347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532b6f7-f4a8-4c38-bd5f-1ee2102790d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:33.701628 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532b6f7-f4a8-4c38-bd5f-1ee2102790d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:33.706268 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6075a-d859-45b7-9a37-f0bcb0c658a3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:33.708973 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3532b6f7-f4a8-4c38-bd5f-1ee2102790d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:33.711154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3532b6f7-f4a8-4c38-bd5f-1ee2102790d7 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:33.723575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-3532b6f7-f4a8-4c38-bd5f-1ee2102790d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:33.723575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:33.723575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:33.723575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:33.723575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:33.723575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:33.723575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:33.723575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:33.723575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:33.723575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:33.723575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:33.723575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:33.723575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:33.723575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:33.723575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:33.724645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:33.724645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:33.724645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:33.724645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:33.724645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:33.724645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:33.724645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:33.724645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:33.724645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:33.724645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:33.724645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:33.724645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:33.724645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:33.724645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:33.724645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:33.724645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:33.724645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:33.731193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:33.731193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:33.731193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:33.731193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:33.731193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:33.731193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:33.731193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:33.731193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:33.731193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 425/860] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:33.731193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532b6f7-f4a8-4c38-bd5f-1ee2102790d7 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:33.747999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453f6ed-ffd3-4896-8505-3c211e35f85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:33.748789 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453f6ed-ffd3-4896-8505-3c211e35f85e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:33.749289 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453f6ed-ffd3-4896-8505-3c211e35f85e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:33.749627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453f6ed-ffd3-4896-8505-3c211e35f85e None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:33.750625 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453f6ed-ffd3-4896-8505-3c211e35f85e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:33.751028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453f6ed-ffd3-4896-8505-3c211e35f85e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:33.752559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453f6ed-ffd3-4896-8505-3c211e35f85e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:33.765151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 436/861] 10.222.0.100 () {56 vars in 1131 bytes} [Tue Aug 2 08:58:33 2022] GET /identity/v3/services?type=compute_legacy => generated 132 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:33.765151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453f6ed-ffd3-4896-8505-3c211e35f85e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:33.848510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:33.855640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532b6f7-f4a8-4c38-bd5f-1ee2102790d7 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:33.894657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1674279-19c6-4aef-9908-d9a42c3453cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:33.895543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1674279-19c6-4aef-9908-d9a42c3453cf None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:33.895667 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1674279-19c6-4aef-9908-d9a42c3453cf None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:33.895864 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1674279-19c6-4aef-9908-d9a42c3453cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:33.909087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1674279-19c6-4aef-9908-d9a42c3453cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:33.910595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1674279-19c6-4aef-9908-d9a42c3453cf None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:33.917395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453f6ed-ffd3-4896-8505-3c211e35f85e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:33.932251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-a1674279-19c6-4aef-9908-d9a42c3453cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:33.932251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:33.932251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:33.932251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:33.932251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:33.932251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:33.932251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:33.932251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:33.932251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:33.932251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:33.932251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:33.932251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:33.932251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:33.932251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:33.932251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:33.934337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:33.934337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:33.934337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:33.934337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:33.934337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:33.934337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:33.934337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:33.934337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:33.934337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:33.934337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:33.934337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:33.934337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:33.934337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:33.934337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:33.934337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:33.934337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:33.934337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:33.935457 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:33.935457 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:33.935457 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:33.935457 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:33.937184 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 426/862] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:33.941785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1674279-19c6-4aef-9908-d9a42c3453cf None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:33.967841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4d6fa-452a-44d3-a47b-de475ffd095b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:33.968483 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4d6fa-452a-44d3-a47b-de475ffd095b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:33.968766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4d6fa-452a-44d3-a47b-de475ffd095b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:33.969044 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4d6fa-452a-44d3-a47b-de475ffd095b None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:33.969844 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4d6fa-452a-44d3-a47b-de475ffd095b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:33.970854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4d6fa-452a-44d3-a47b-de475ffd095b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:33.972648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4d6fa-452a-44d3-a47b-de475ffd095b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:33.981126 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:33.992978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 437/863] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:34.002083 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4d6fa-452a-44d3-a47b-de475ffd095b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:34.081799 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1674279-19c6-4aef-9908-d9a42c3453cf None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:34.120483 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4d6fa-452a-44d3-a47b-de475ffd095b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:34.122561 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720879f9-80fe-4ed9-856d-6208b2c76206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:34.123499 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720879f9-80fe-4ed9-856d-6208b2c76206 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:34.123931 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720879f9-80fe-4ed9-856d-6208b2c76206 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:34.124355 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720879f9-80fe-4ed9-856d-6208b2c76206 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:34.125468 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720879f9-80fe-4ed9-856d-6208b2c76206 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:34.125919 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720879f9-80fe-4ed9-856d-6208b2c76206 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:34.127431 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720879f9-80fe-4ed9-856d-6208b2c76206 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:34.145275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 427/864] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:34.145825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:34.147967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720879f9-80fe-4ed9-856d-6208b2c76206 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:34.159198 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa4460-88d9-4d6a-954b-7360bd2201b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:34.159825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa4460-88d9-4d6a-954b-7360bd2201b7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:34.160111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa4460-88d9-4d6a-954b-7360bd2201b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:34.160390 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa4460-88d9-4d6a-954b-7360bd2201b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:34.161224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa4460-88d9-4d6a-954b-7360bd2201b7 None admin] RBAC: Adding query filter params (role.id=7cbff51867a7475d837afd51f6c12f11, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=50cb532b58ed4951ba960a0a7d579b49) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:34.161519 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa4460-88d9-4d6a-954b-7360bd2201b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:34.163661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa4460-88d9-4d6a-954b-7360bd2201b7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:34.177101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 438/865] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:58:34 2022] GET /identity/v3/role_assignments?user.id=50cb532b58ed4951ba960a0a7d579b49&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=7cbff51867a7475d837afd51f6c12f11 => generated 610 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:34.260238 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:34.273102 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720879f9-80fe-4ed9-856d-6208b2c76206 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:34.312497 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d0bc2-8028-466a-bcb4-8dbbb10e2159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:34.313283 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d0bc2-8028-466a-bcb4-8dbbb10e2159 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:34.313643 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d0bc2-8028-466a-bcb4-8dbbb10e2159 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:34.313994 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d0bc2-8028-466a-bcb4-8dbbb10e2159 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:34.322521 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d0bc2-8028-466a-bcb4-8dbbb10e2159 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:34.324225 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d0bc2-8028-466a-bcb4-8dbbb10e2159 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:34.338417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-134d0bc2-8028-466a-bcb4-8dbbb10e2159 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:58:34.338417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:34.338417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:34.338417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:34.338417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:34.338417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:34.338417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:34.338417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:34.338417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:34.338417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:34.338417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:34.338417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:34.338417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:34.338417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:34.338417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:34.338417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:34.339595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:34.339595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:34.339595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:34.339595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:34.339595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:34.339595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:34.339595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:34.339595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:34.339595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:34.339595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:34.339595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:34.339595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:34.339595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:34.339595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:34.339595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:34.339595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:34.339595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:34.339595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:34.340938 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:34.340938 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:34.340938 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:34.340938 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:34.340938 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:34.340938 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:34.340938 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:34.340938 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:34.340938 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:58:34.340938 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:34.343242 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 428/866] 10.222.0.100 () {56 vars in 1122 bytes} [Tue Aug 2 08:58:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:34.350841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa4460-88d9-4d6a-954b-7360bd2201b7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:34.457911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa4460-88d9-4d6a-954b-7360bd2201b7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:34.495007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df159f-940a-49b0-b873-a1c30c408371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:34.495913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df159f-940a-49b0-b873-a1c30c408371 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:34.496337 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df159f-940a-49b0-b873-a1c30c408371 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:34.496996 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df159f-940a-49b0-b873-a1c30c408371 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:34.498040 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df159f-940a-49b0-b873-a1c30c408371 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:34.498516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df159f-940a-49b0-b873-a1c30c408371 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:34.500336 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df159f-940a-49b0-b873-a1c30c408371 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:34.520174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 439/867] 10.222.0.100 () {56 vars in 1154 bytes} [Tue Aug 2 08:58:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:34.526833 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d0bc2-8028-466a-bcb4-8dbbb10e2159 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:34.637009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d0bc2-8028-466a-bcb4-8dbbb10e2159 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:34.672837 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83427258-f960-4c6c-b4a8-45d876a96948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:34.673793 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83427258-f960-4c6c-b4a8-45d876a96948 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:34.674217 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83427258-f960-4c6c-b4a8-45d876a96948 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:34.674644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83427258-f960-4c6c-b4a8-45d876a96948 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:34.694701 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:34.747040 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83427258-f960-4c6c-b4a8-45d876a96948 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:34.748732 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83427258-f960-4c6c-b4a8-45d876a96948 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:34.852304 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 429/868] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:58:34 2022] GET /identity/v3/auth/tokens => generated 1264 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:58:34.858129 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df159f-940a-49b0-b873-a1c30c408371 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:34.879174 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f1296-b9d5-4a74-be74-ac153ebaebf7 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:34.879518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f1296-b9d5-4a74-be74-ac153ebaebf7 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:34.879805 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f1296-b9d5-4a74-be74-ac153ebaebf7 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:34.881576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 430/869] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:58:34.886850 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb16d74-d3bf-48d6-bb87-4327b8cf525a None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:34.887140 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb16d74-d3bf-48d6-bb87-4327b8cf525a None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:34.887423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb16d74-d3bf-48d6-bb87-4327b8cf525a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:34.909310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-cfb16d74-d3bf-48d6-bb87-4327b8cf525a None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:34.910809 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-cfb16d74-d3bf-48d6-bb87-4327b8cf525a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:34.912029 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:34.977840 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:34.981625 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df159f-940a-49b0-b873-a1c30c408371 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:35.012415 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 431/870] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:34 2022] POST /identity/v3/auth/tokens => generated 1264 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:58:35.020128 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7206416-1c19-4e97-bf53-d2f5b7d2c852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:35.021083 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7206416-1c19-4e97-bf53-d2f5b7d2c852 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:35.021651 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7206416-1c19-4e97-bf53-d2f5b7d2c852 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:35.022658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7206416-1c19-4e97-bf53-d2f5b7d2c852 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:35.034393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb16d74-d3bf-48d6-bb87-4327b8cf525a None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:35.038443 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7206416-1c19-4e97-bf53-d2f5b7d2c852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:35.042694 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7206416-1c19-4e97-bf53-d2f5b7d2c852 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:35.062617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-f7206416-1c19-4e97-bf53-d2f5b7d2c852 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:35.062617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:35.062617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:35.062617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:35.062617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:35.062617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:35.062617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:35.062617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:35.062617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:35.062617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:35.062617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:35.062617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:35.062617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:35.062617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:35.062617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:35.063591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:35.063591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:35.063591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:35.063591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:35.063591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:35.063591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:35.063591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:35.063591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:35.063591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:35.063591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:35.063591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:35.063591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:35.063591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:35.063591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:35.063591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:35.063591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:35.063591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:35.064475 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:35.064475 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:35.064475 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:35.064475 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:35.064475 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 440/871] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:35.069272 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7206416-1c19-4e97-bf53-d2f5b7d2c852 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:35.148767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb16d74-d3bf-48d6-bb87-4327b8cf525a None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:35.184425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834b235-5801-4739-84ac-e44d82a3d6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:35.185319 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834b235-5801-4739-84ac-e44d82a3d6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:35.185432 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834b235-5801-4739-84ac-e44d82a3d6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:35.185698 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834b235-5801-4739-84ac-e44d82a3d6a5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:35.186313 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834b235-5801-4739-84ac-e44d82a3d6a5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:35.187682 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834b235-5801-4739-84ac-e44d82a3d6a5 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:35.192687 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:35.198087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-3834b235-5801-4739-84ac-e44d82a3d6a5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 08:58:35.198087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:35.198087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:35.198087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:35.198087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:35.198087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:35.198087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:35.198087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:35.198087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 08:58:35.198087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:35.198087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:35.198087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:35.198087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:35.198087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:35.198087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:35.198087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:35.198087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:35.199068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:35.199068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:35.199068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:35.199068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:35.199068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:35.199068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:35.199068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:35.199068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:35.199068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:35.199068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:35.199068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:35.199068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:35.199068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:35.199068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:35.199068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:35.199068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:35.199918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:35.199918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:35.199918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:35.199918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:35.199918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:35.199918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:35.199918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:35.199918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:35.199918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:35.199918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:35.199918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:35.199918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:35.199918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 08:58:35.199918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:35.199918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 432/872] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:58:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:35.204803 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834b235-5801-4739-84ac-e44d82a3d6a5 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:35.211954 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7206416-1c19-4e97-bf53-d2f5b7d2c852 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:35.248618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f88bc-b5ee-49f7-92a5-c0352e719dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:35.249291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f88bc-b5ee-49f7-92a5-c0352e719dcb None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:35.249577 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f88bc-b5ee-49f7-92a5-c0352e719dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:35.250118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f88bc-b5ee-49f7-92a5-c0352e719dcb None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:35.250807 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f88bc-b5ee-49f7-92a5-c0352e719dcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:35.251118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f88bc-b5ee-49f7-92a5-c0352e719dcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:35.252477 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f88bc-b5ee-49f7-92a5-c0352e719dcb None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:35.260063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:35.272992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 441/873] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:35.274544 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f88bc-b5ee-49f7-92a5-c0352e719dcb None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:35.332697 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834b235-5801-4739-84ac-e44d82a3d6a5 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:35.376657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eede4f-079c-4b12-9d7a-557f81b864ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:35.378161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eede4f-079c-4b12-9d7a-557f81b864ce None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:35.378649 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eede4f-079c-4b12-9d7a-557f81b864ce None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:35.379142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eede4f-079c-4b12-9d7a-557f81b864ce None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:35.380442 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eede4f-079c-4b12-9d7a-557f81b864ce None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:35.381209 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eede4f-079c-4b12-9d7a-557f81b864ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:35.383601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eede4f-079c-4b12-9d7a-557f81b864ce None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:35.386824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f88bc-b5ee-49f7-92a5-c0352e719dcb None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:35.394310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 433/874] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:58:35 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:35.395958 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eede4f-079c-4b12-9d7a-557f81b864ce None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:35.424534 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613ad5c-4ebf-4958-8463-c05e3f9240d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:35.425234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613ad5c-4ebf-4958-8463-c05e3f9240d3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:35.425538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613ad5c-4ebf-4958-8463-c05e3f9240d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:35.425796 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613ad5c-4ebf-4958-8463-c05e3f9240d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:35.433666 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613ad5c-4ebf-4958-8463-c05e3f9240d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:35.435199 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613ad5c-4ebf-4958-8463-c05e3f9240d3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:35.447798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-9613ad5c-4ebf-4958-8463-c05e3f9240d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:35.447798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:35.447798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:35.447798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:35.447798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:35.447798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:35.447798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:35.447798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:35.447798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:35.447798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:35.447798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:35.447798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:35.447798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:35.447798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:35.447798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:35.448769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:35.448769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:35.448769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:35.448769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:35.448769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:35.448769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:35.448769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:35.448769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:35.448769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:35.448769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:35.448769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:35.448769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:35.448769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:35.448769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:35.448769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:35.448769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:35.448769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:35.449683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:35.449683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:35.449683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:35.449683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:35.449683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:35.449683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:35.449683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:35.449683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:35.453590 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 442/875] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:35.455361 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613ad5c-4ebf-4958-8463-c05e3f9240d3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:35.474168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:35.512544 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eede4f-079c-4b12-9d7a-557f81b864ce None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:35.522928 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:35.552822 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3952a60-16ba-4f52-a410-b58e839d7681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:35.554638 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3952a60-16ba-4f52-a410-b58e839d7681 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:35.555119 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3952a60-16ba-4f52-a410-b58e839d7681 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:35.555616 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3952a60-16ba-4f52-a410-b58e839d7681 None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:35.556493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3952a60-16ba-4f52-a410-b58e839d7681 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:35.557008 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3952a60-16ba-4f52-a410-b58e839d7681 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:35.558610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3952a60-16ba-4f52-a410-b58e839d7681 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:35.561501 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613ad5c-4ebf-4958-8463-c05e3f9240d3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:35.569558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 434/876] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:58:35 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:35.570746 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3952a60-16ba-4f52-a410-b58e839d7681 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:35.598169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac74a5-378d-4b9c-8465-fccf21f7488e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:35.598957 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac74a5-378d-4b9c-8465-fccf21f7488e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:35.599359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac74a5-378d-4b9c-8465-fccf21f7488e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:35.599696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac74a5-378d-4b9c-8465-fccf21f7488e None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:35.601297 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac74a5-378d-4b9c-8465-fccf21f7488e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:35.601765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac74a5-378d-4b9c-8465-fccf21f7488e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:35.603395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac74a5-378d-4b9c-8465-fccf21f7488e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:35.620984 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 443/877] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:35.622413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac74a5-378d-4b9c-8465-fccf21f7488e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:35.677435 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3952a60-16ba-4f52-a410-b58e839d7681 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:35.714092 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b65a79-eefa-458b-a138-df41532d49e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:35.715026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b65a79-eefa-458b-a138-df41532d49e1 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:35.715506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b65a79-eefa-458b-a138-df41532d49e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:35.715962 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b65a79-eefa-458b-a138-df41532d49e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:35.716931 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b65a79-eefa-458b-a138-df41532d49e1 None admin] RBAC: Adding query filter params (interface=public, service_id=c4a8868c8699432cbf4ee5d515fc8780, region_id=RegionOne) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:35.717484 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b65a79-eefa-458b-a138-df41532d49e1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:35.719121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b65a79-eefa-458b-a138-df41532d49e1 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:35.729133 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac74a5-378d-4b9c-8465-fccf21f7488e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:35.729382 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 435/878] 10.222.0.100 () {56 vars in 1256 bytes} [Tue Aug 2 08:58:35 2022] GET /identity/v3/endpoints?service_id=c4a8868c8699432cbf4ee5d515fc8780&interface=public®ion_id=RegionOne => generated 195 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:35.730381 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b65a79-eefa-458b-a138-df41532d49e1 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:35.736671 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:35.765792 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a9b73-2efb-4d36-8add-cd4df51f989e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:35.766536 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a9b73-2efb-4d36-8add-cd4df51f989e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:35.766887 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a9b73-2efb-4d36-8add-cd4df51f989e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:35.767232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a9b73-2efb-4d36-8add-cd4df51f989e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:35.768087 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a9b73-2efb-4d36-8add-cd4df51f989e None admin] RBAC: Adding query filter params (role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=1cd36b543bd9487ebd8ce578e9a76c78) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:35.768510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a9b73-2efb-4d36-8add-cd4df51f989e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:35.770039 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a9b73-2efb-4d36-8add-cd4df51f989e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:35.770423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:35.785298 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 444/879] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:58:35 2022] GET /identity/v3/role_assignments?user.id=1cd36b543bd9487ebd8ce578e9a76c78&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 610 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:35.850787 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b65a79-eefa-458b-a138-df41532d49e1 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:35.852888 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cb1b3-79f2-4117-8c96-f2dd1274ba53 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:35.853211 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cb1b3-79f2-4117-8c96-f2dd1274ba53 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:35.853563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cb1b3-79f2-4117-8c96-f2dd1274ba53 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:35.855297 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 445/880] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:58:35.860849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6bdc7-8994-41be-bd3d-458c08a69d90 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:35.861033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6bdc7-8994-41be-bd3d-458c08a69d90 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:35.861365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6bdc7-8994-41be-bd3d-458c08a69d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:35.880956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e7c6bdc7-8994-41be-bd3d-458c08a69d90 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:35.882892 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e7c6bdc7-8994-41be-bd3d-458c08a69d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:35.892952 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d468da1-d7f8-4eff-ace5-32c0a662f18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:35.893915 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d468da1-d7f8-4eff-ace5-32c0a662f18e None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:35.894372 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d468da1-d7f8-4eff-ace5-32c0a662f18e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:35.894845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d468da1-d7f8-4eff-ace5-32c0a662f18e None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:35.895674 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d468da1-d7f8-4eff-ace5-32c0a662f18e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:35.897302 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d468da1-d7f8-4eff-ace5-32c0a662f18e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:35.904211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 436/881] 10.222.0.100 () {56 vars in 1092 bytes} [Tue Aug 2 08:58:35 2022] GET /identity/v3/services => generated 1016 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 08:58:35.968542 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 446/882] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:35 2022] POST /identity/v3/auth/tokens => generated 1264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:58:35.991796 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d468da1-d7f8-4eff-ace5-32c0a662f18e None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:36.105081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:36.109083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d468da1-d7f8-4eff-ace5-32c0a662f18e None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:36.146599 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f778f0-209b-4a4d-a544-b6a9136eacbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:36.148117 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f778f0-209b-4a4d-a544-b6a9136eacbf None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:36.148620 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f778f0-209b-4a4d-a544-b6a9136eacbf None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:36.149085 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f778f0-209b-4a4d-a544-b6a9136eacbf None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:36.150044 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f778f0-209b-4a4d-a544-b6a9136eacbf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:36.151620 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f778f0-209b-4a4d-a544-b6a9136eacbf None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:36.169361 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 437/883] 10.222.0.100 () {60 vars in 1146 bytes} [Tue Aug 2 08:58:35 2022] POST /identity/v3/services => generated 270 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:58:36.395519 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714afdd-8845-45b8-92bd-0e2dcb7a9f94 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:36.395863 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714afdd-8845-45b8-92bd-0e2dcb7a9f94 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:36.396043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714afdd-8845-45b8-92bd-0e2dcb7a9f94 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:36.398059 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 447/884] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:36 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:36.403361 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b4652-0f50-4afe-a048-2745b3f24013 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:36.403913 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b4652-0f50-4afe-a048-2745b3f24013 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:36.404393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b4652-0f50-4afe-a048-2745b3f24013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:36.423822 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-e35b4652-0f50-4afe-a048-2745b3f24013 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:36.425371 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e35b4652-0f50-4afe-a048-2745b3f24013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:36.502709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 438/885] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:36 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:36.526077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714afdd-8845-45b8-92bd-0e2dcb7a9f94 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:36.600124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:36.638118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714afdd-8845-45b8-92bd-0e2dcb7a9f94 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:36.675702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25460a4-9253-4af7-8640-15886664b4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:36.676316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25460a4-9253-4af7-8640-15886664b4e7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:36.676603 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25460a4-9253-4af7-8640-15886664b4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:36.676876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25460a4-9253-4af7-8640-15886664b4e7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:36.677550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25460a4-9253-4af7-8640-15886664b4e7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:36.679697 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25460a4-9253-4af7-8640-15886664b4e7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:36.689234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-b25460a4-9253-4af7-8640-15886664b4e7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:58:36.689234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:36.689234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:36.689234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:36.689234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:36.689234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:36.689234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:36.689234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:36.689234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:58:36.689234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:36.689234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:36.689234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:36.689234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:36.689234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:36.689234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:36.689234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:36.689234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:36.690260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:36.690260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:36.690260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:36.690260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:36.690260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:36.690260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:36.690260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:36.690260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:36.690260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:36.690260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:36.690260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:36.690260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:36.690260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:36.690260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:36.690260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:36.690260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:36.691113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:36.691113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:36.691113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:36.691113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:36.691113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:36.691113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:36.691113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:36.691113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:36.691113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:36.691113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:36.691113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:36.691113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:36.691113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:58:36.691113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:36.691113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 448/886] 10.222.0.100 () {56 vars in 1131 bytes} [Tue Aug 2 08:58:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:36.698783 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b4652-0f50-4afe-a048-2745b3f24013 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:36.786102 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:36.805156 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b4652-0f50-4afe-a048-2745b3f24013 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:36.841310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e65e5-61ff-42e7-b94f-c20e81ee029a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:36.842276 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e65e5-61ff-42e7-b94f-c20e81ee029a None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:36.842763 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e65e5-61ff-42e7-b94f-c20e81ee029a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:36.843260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e65e5-61ff-42e7-b94f-c20e81ee029a None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:36.844150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365e65e5-61ff-42e7-b94f-c20e81ee029a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:36.844660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365e65e5-61ff-42e7-b94f-c20e81ee029a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:36.846257 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365e65e5-61ff-42e7-b94f-c20e81ee029a None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:36.852162 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 439/887] 10.222.0.100 () {56 vars in 1127 bytes} [Tue Aug 2 08:58:36 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:36.858117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25460a4-9253-4af7-8640-15886664b4e7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:36.967606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25460a4-9253-4af7-8640-15886664b4e7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:36.997641 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:37.002829 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5dbef-f357-4baf-8f32-e2f36b873e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:37.003454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5dbef-f357-4baf-8f32-e2f36b873e95 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:37.003739 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5dbef-f357-4baf-8f32-e2f36b873e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:37.004016 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5dbef-f357-4baf-8f32-e2f36b873e95 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:37.004700 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5dbef-f357-4baf-8f32-e2f36b873e95 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:37.005008 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5dbef-f357-4baf-8f32-e2f36b873e95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:37.006462 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5dbef-f357-4baf-8f32-e2f36b873e95 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:37.011570 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 449/888] 10.222.0.100 () {56 vars in 1127 bytes} [Tue Aug 2 08:58:36 2022] GET /identity/v3/services?type=object-store => generated 130 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:38.005092 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324faf6f-ec18-4b01-9fea-73c0cc22d4ba None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:38.005619 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324faf6f-ec18-4b01-9fea-73c0cc22d4ba None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:38.005696 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324faf6f-ec18-4b01-9fea-73c0cc22d4ba None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:38.007750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 440/889] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:38 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:38.012896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152394ee-3285-4017-80a5-2f40e7f2e8b0 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:38.013350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152394ee-3285-4017-80a5-2f40e7f2e8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:38.013525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152394ee-3285-4017-80a5-2f40e7f2e8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:38.019116 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc9a65-5389-4ed9-9a2a-8d229f98f0e7 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:38.019626 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc9a65-5389-4ed9-9a2a-8d229f98f0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:38.020080 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc9a65-5389-4ed9-9a2a-8d229f98f0e7 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:38.022522 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 441/890] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:38 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:58:38.027544 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8964d-6c00-4a5d-a713-c7ff31e30823 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:38.028061 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8964d-6c00-4a5d-a713-c7ff31e30823 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:38.028577 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8964d-6c00-4a5d-a713-c7ff31e30823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:38.032761 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-152394ee-3285-4017-80a5-2f40e7f2e8b0 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:38.034187 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-152394ee-3285-4017-80a5-2f40e7f2e8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:38.047288 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-b2a8964d-6c00-4a5d-a713-c7ff31e30823 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:38.048573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b2a8964d-6c00-4a5d-a713-c7ff31e30823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:38.121262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 450/891] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:38 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:38.129375 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 442/892] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:38 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:38.144782 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8964d-6c00-4a5d-a713-c7ff31e30823 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:38.145846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152394ee-3285-4017-80a5-2f40e7f2e8b0 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:38.236224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:38.258961 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8964d-6c00-4a5d-a713-c7ff31e30823 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:38.280135 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152394ee-3285-4017-80a5-2f40e7f2e8b0 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:38.296561 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702707f-4bba-4c86-bfba-8a34e6297e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:38.297466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702707f-4bba-4c86-bfba-8a34e6297e6e None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:38.298063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702707f-4bba-4c86-bfba-8a34e6297e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:38.298807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702707f-4bba-4c86-bfba-8a34e6297e6e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:38.299620 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702707f-4bba-4c86-bfba-8a34e6297e6e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:38.301697 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702707f-4bba-4c86-bfba-8a34e6297e6e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:38.307492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:38.314243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-3702707f-4bba-4c86-bfba-8a34e6297e6e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 08:58:38.314243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:38.314243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:38.314243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:38.314243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:38.314243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:38.314243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:38.314243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:38.314243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 08:58:38.314243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:38.314243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:38.314243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:38.314243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:38.314243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:38.314243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:38.314243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:38.314243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:38.315393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:38.315393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:38.315393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:38.315393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:38.315393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:38.315393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:38.315393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:38.315393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:38.315393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:38.315393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:38.315393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:38.315393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:38.315393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:38.315393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:38.315393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:38.315393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:38.316420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:38.316420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:38.316420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:38.316420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:38.316420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:38.316420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:38.316420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:38.316420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:38.316420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:38.316420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:38.316420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:38.316420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:38.316420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 08:58:38.316420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:38.317364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400ede5-359c-44cf-825e-9fc891587fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:38.317364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400ede5-359c-44cf-825e-9fc891587fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:38.317364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400ede5-359c-44cf-825e-9fc891587fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:38.317607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400ede5-359c-44cf-825e-9fc891587fc4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:38.319462 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 443/893] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:58:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:38.320919 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d039511-c17a-4876-97f2-d1acab1860f4 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:38.321559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d039511-c17a-4876-97f2-d1acab1860f4 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:38.321990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d039511-c17a-4876-97f2-d1acab1860f4 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:38.327623 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 444/894] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:38 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:38.327623 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d039511-c17a-4876-97f2-d1acab1860f4 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:38.332067 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400ede5-359c-44cf-825e-9fc891587fc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:38.338401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-d400ede5-359c-44cf-825e-9fc891587fc4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:58:38.338401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:38.338401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:38.338401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:38.338401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:38.338401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:38.338401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:38.338401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:38.338401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:38.338401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:38.338401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:38.338401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:38.338401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:38.338401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:38.338401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:38.338401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:38.339330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:38.339330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:38.339330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:38.339330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:38.339330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:38.339330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:38.339330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:38.339330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:38.339330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:38.339330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:38.339330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:38.339330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:38.339330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:38.339330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:38.339330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:38.339330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:38.340207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:38.340207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:38.340207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:38.340207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:38.340207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:38.340207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:38.340207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:38.340207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:38.340207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:38.340207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:38.340207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:58:38.340207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:38.341730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 451/895] 10.222.0.100 () {56 vars in 1125 bytes} [Tue Aug 2 08:58:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:38.342943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c440e6-324d-40b5-bfef-c04ec24becd1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:38.343292 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c440e6-324d-40b5-bfef-c04ec24becd1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:38.343581 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c440e6-324d-40b5-bfef-c04ec24becd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:38.363299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-38c440e6-324d-40b5-bfef-c04ec24becd1 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:38.364486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-38c440e6-324d-40b5-bfef-c04ec24becd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:38.436853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d039511-c17a-4876-97f2-d1acab1860f4 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:38.449134 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 452/896] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:38 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:38.449134 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c440e6-324d-40b5-bfef-c04ec24becd1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:38.474408 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6dd02a-b461-48e6-b46b-43ce8ac27d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:38.475563 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6dd02a-b461-48e6-b46b-43ce8ac27d3d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:38.475841 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6dd02a-b461-48e6-b46b-43ce8ac27d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:38.476122 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6dd02a-b461-48e6-b46b-43ce8ac27d3d None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:38.476800 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6dd02a-b461-48e6-b46b-43ce8ac27d3d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:38.477142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6dd02a-b461-48e6-b46b-43ce8ac27d3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:38.478512 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6dd02a-b461-48e6-b46b-43ce8ac27d3d None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:38.483624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 445/897] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:58:38 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:38.484241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6dd02a-b461-48e6-b46b-43ce8ac27d3d None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:38.496915 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:38.567041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c440e6-324d-40b5-bfef-c04ec24becd1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:38.585905 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:38.596436 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6dd02a-b461-48e6-b46b-43ce8ac27d3d None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:38.605654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc063e5-739e-4c84-bf57-e504649e6902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:38.606283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc063e5-739e-4c84-bf57-e504649e6902 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:38.606563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc063e5-739e-4c84-bf57-e504649e6902 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:38.606836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc063e5-739e-4c84-bf57-e504649e6902 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:38.607546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc063e5-739e-4c84-bf57-e504649e6902 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:38.607850 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc063e5-739e-4c84-bf57-e504649e6902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:38.609230 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc063e5-739e-4c84-bf57-e504649e6902 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:38.622786 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 453/898] 10.222.0.100 () {56 vars in 1120 bytes} [Tue Aug 2 08:58:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:38.623658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc063e5-739e-4c84-bf57-e504649e6902 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:38.634180 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fdf8a-3f02-4b56-96ee-a80aaf5db6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:38.634836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fdf8a-3f02-4b56-96ee-a80aaf5db6ec None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:38.635123 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fdf8a-3f02-4b56-96ee-a80aaf5db6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:38.635402 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fdf8a-3f02-4b56-96ee-a80aaf5db6ec None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:38.636033 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65fdf8a-3f02-4b56-96ee-a80aaf5db6ec None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:38.637424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65fdf8a-3f02-4b56-96ee-a80aaf5db6ec None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:38.646548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-e65fdf8a-3f02-4b56-96ee-a80aaf5db6ec None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 08:58:38.646548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:38.646548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:38.646548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:38.646548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:38.646548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:38.646548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:38.646548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:38.646548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 08:58:38.646548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:38.646548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:38.646548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:38.646548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:38.646548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:38.646548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:38.646548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:38.648325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:38.648325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:38.648325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:38.648325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:38.648325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:38.648325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:38.648325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:38.648325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:38.648325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:38.648325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:38.648325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:38.648325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:38.648325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:38.648325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:38.648325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:38.648325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:38.649226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:38.649226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:38.649226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:38.649226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:38.649226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:38.649226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:38.649226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:38.649226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:38.649226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:38.649226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:38.649226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:38.649226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:38.649226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:38.649226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 08:58:38.649226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:38.649226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 446/899] 10.222.0.100 () {56 vars in 1137 bytes} [Tue Aug 2 08:58:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:38.650581 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fdf8a-3f02-4b56-96ee-a80aaf5db6ec None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:38.724482 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:38.748254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc063e5-739e-4c84-bf57-e504649e6902 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:38.760096 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fdf8a-3f02-4b56-96ee-a80aaf5db6ec None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:38.784944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4d653-8696-44f6-9378-dd8d50a016f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:38.785611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4d653-8696-44f6-9378-dd8d50a016f1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:38.785895 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4d653-8696-44f6-9378-dd8d50a016f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:38.786172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4d653-8696-44f6-9378-dd8d50a016f1 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:38.786855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4d653-8696-44f6-9378-dd8d50a016f1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:38.787167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4d653-8696-44f6-9378-dd8d50a016f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:38.789734 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4d653-8696-44f6-9378-dd8d50a016f1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:38.794704 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 454/900] 10.222.0.100 () {56 vars in 1119 bytes} [Tue Aug 2 08:58:38 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:38.795544 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4d653-8696-44f6-9378-dd8d50a016f1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:38.802388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f3fa6-22ce-4793-89df-d2252268f45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:38.804384 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f3fa6-22ce-4793-89df-d2252268f45d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:38.804672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f3fa6-22ce-4793-89df-d2252268f45d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:38.804980 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f3fa6-22ce-4793-89df-d2252268f45d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:38.817675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f3fa6-22ce-4793-89df-d2252268f45d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:38.819698 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f3fa6-22ce-4793-89df-d2252268f45d None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:38.834894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-e62f3fa6-22ce-4793-89df-d2252268f45d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:38.834894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:38.834894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:38.834894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:38.834894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:38.834894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:38.834894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:38.834894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:38.834894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:38.834894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:38.834894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:38.834894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:38.834894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:38.834894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:38.834894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:38.835989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:38.835989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:38.835989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:38.835989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:38.835989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:38.835989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:38.835989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:38.835989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:38.835989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:38.835989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:38.835989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:38.835989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:38.835989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:38.835989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:38.835989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:38.835989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:38.835989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:38.836951 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:38.836951 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:38.836951 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:38.836951 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:38.836951 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 447/901] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:38.837583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f3fa6-22ce-4793-89df-d2252268f45d None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:38.852622 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:38.921794 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4d653-8696-44f6-9378-dd8d50a016f1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:38.954692 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f3fa6-22ce-4793-89df-d2252268f45d None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:38.957061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ec4dc-ef0f-4363-8147-4ced7bce8989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:38.957711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ec4dc-ef0f-4363-8147-4ced7bce8989 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:38.957990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ec4dc-ef0f-4363-8147-4ced7bce8989 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:38.958264 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ec4dc-ef0f-4363-8147-4ced7bce8989 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:38.958938 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ec4dc-ef0f-4363-8147-4ced7bce8989 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:38.960027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ec4dc-ef0f-4363-8147-4ced7bce8989 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:38.961861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ec4dc-ef0f-4363-8147-4ced7bce8989 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:38.966972 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 455/902] 10.222.0.100 () {56 vars in 1131 bytes} [Tue Aug 2 08:58:38 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:38.967566 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ec4dc-ef0f-4363-8147-4ced7bce8989 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:38.985483 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:38.999388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac268b-cd10-4bd8-a6fe-e57f621893ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:38.999388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac268b-cd10-4bd8-a6fe-e57f621893ab None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:38.999388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac268b-cd10-4bd8-a6fe-e57f621893ab None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:38.999388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac268b-cd10-4bd8-a6fe-e57f621893ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:38.999983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac268b-cd10-4bd8-a6fe-e57f621893ab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:39.000668 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac268b-cd10-4bd8-a6fe-e57f621893ab None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:39.030920 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 448/903] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 08:58:38 2022] POST /identity/v3/endpoints => generated 348 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:58:39.031873 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac268b-cd10-4bd8-a6fe-e57f621893ab None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:39.096747 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ec4dc-ef0f-4363-8147-4ced7bce8989 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:39.105415 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:39.140881 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074b64f-ed9b-4164-b86f-4a1536a76252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:39.141547 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074b64f-ed9b-4164-b86f-4a1536a76252 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:39.141834 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074b64f-ed9b-4164-b86f-4a1536a76252 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:39.142110 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074b64f-ed9b-4164-b86f-4a1536a76252 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:39.142821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074b64f-ed9b-4164-b86f-4a1536a76252 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:39.143132 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074b64f-ed9b-4164-b86f-4a1536a76252 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:39.144531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074b64f-ed9b-4164-b86f-4a1536a76252 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:39.157782 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 456/904] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:39.158591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac268b-cd10-4bd8-a6fe-e57f621893ab None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:39.159091 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab19d7-609a-47d6-90e2-88d30ab2d9b1 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:39.159388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab19d7-609a-47d6-90e2-88d30ab2d9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:39.159673 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab19d7-609a-47d6-90e2-88d30ab2d9b1 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:39.165235 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 457/905] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:39 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:39.166171 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab19d7-609a-47d6-90e2-88d30ab2d9b1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:39.203874 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe74ce-0314-49c9-94b8-bafa310e9dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:39.204659 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe74ce-0314-49c9-94b8-bafa310e9dbf None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:39.204945 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe74ce-0314-49c9-94b8-bafa310e9dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:39.205268 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe74ce-0314-49c9-94b8-bafa310e9dbf None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:39.205962 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe74ce-0314-49c9-94b8-bafa310e9dbf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:39.206269 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe74ce-0314-49c9-94b8-bafa310e9dbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:39.207659 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe74ce-0314-49c9-94b8-bafa310e9dbf None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:39.213653 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 449/906] 10.222.0.100 () {56 vars in 1131 bytes} [Tue Aug 2 08:58:39 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:39.214659 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f66b4a-7cb4-45bf-bd5f-0104dd4af4bd None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:39.214948 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f66b4a-7cb4-45bf-bd5f-0104dd4af4bd None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:39.215220 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f66b4a-7cb4-45bf-bd5f-0104dd4af4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:39.234196 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-90f66b4a-7cb4-45bf-bd5f-0104dd4af4bd None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:39.235398 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-90f66b4a-7cb4-45bf-bd5f-0104dd4af4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:39.278155 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:39.284226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab19d7-609a-47d6-90e2-88d30ab2d9b1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:39.309523 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 450/907] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:39 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:39.310376 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f66b4a-7cb4-45bf-bd5f-0104dd4af4bd None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:39.328399 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361283c8-1ea4-487d-bda0-64621f785447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:39.330181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361283c8-1ea4-487d-bda0-64621f785447 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:39.330676 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361283c8-1ea4-487d-bda0-64621f785447 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:39.331419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361283c8-1ea4-487d-bda0-64621f785447 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:39.342318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361283c8-1ea4-487d-bda0-64621f785447 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:39.343759 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361283c8-1ea4-487d-bda0-64621f785447 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:39.356214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-361283c8-1ea4-487d-bda0-64621f785447 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:58:39.356214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:39.356214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:39.356214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:39.356214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:39.356214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:39.356214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:39.356214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:39.356214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:39.356214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:39.356214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:39.356214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:39.356214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:39.356214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:39.356214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:39.356214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:39.357487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:39.357487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:39.357487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:39.357487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:39.357487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:39.357487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:39.357487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:39.357487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:39.357487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:39.357487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:39.357487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:39.357487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:39.357487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:39.357487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:39.357487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:39.357487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:39.357487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:39.357487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:39.359101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:39.359101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:39.359101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:39.359101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:39.359101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:39.359101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:39.359101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:39.359101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:39.359101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:58:39.359101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:39.362297 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 458/908] 10.222.0.100 () {56 vars in 1122 bytes} [Tue Aug 2 08:58:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:39.362969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361283c8-1ea4-487d-bda0-64621f785447 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:39.393860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:39.428976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f66b4a-7cb4-45bf-bd5f-0104dd4af4bd None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:39.466048 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670ae3d-af89-4102-a1cc-a3e0f70e33df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:39.466916 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670ae3d-af89-4102-a1cc-a3e0f70e33df None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:39.467200 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670ae3d-af89-4102-a1cc-a3e0f70e33df None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:39.467475 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670ae3d-af89-4102-a1cc-a3e0f70e33df None admin] PATH_INFO: `/v3/endpoints` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:39.468224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670ae3d-af89-4102-a1cc-a3e0f70e33df None admin] RBAC: Adding query filter params (interface=public, service_id=04c332c8b87b4ea9881c83c504caa3c5, region_id=RegionOne) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:39.468533 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670ae3d-af89-4102-a1cc-a3e0f70e33df None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:39.469945 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670ae3d-af89-4102-a1cc-a3e0f70e33df None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:39.480617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 451/909] 10.222.0.100 () {56 vars in 1256 bytes} [Tue Aug 2 08:58:39 2022] GET /identity/v3/endpoints?service_id=04c332c8b87b4ea9881c83c504caa3c5&interface=public®ion_id=RegionOne => generated 195 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:39.480617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670ae3d-af89-4102-a1cc-a3e0f70e33df None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:39.480883 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361283c8-1ea4-487d-bda0-64621f785447 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:39.517388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217cda8-1881-42e0-a45a-2222437ad6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:39.518014 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217cda8-1881-42e0-a45a-2222437ad6f9 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:39.518297 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217cda8-1881-42e0-a45a-2222437ad6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:39.518576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217cda8-1881-42e0-a45a-2222437ad6f9 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:39.519273 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7217cda8-1881-42e0-a45a-2222437ad6f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:39.520637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7217cda8-1881-42e0-a45a-2222437ad6f9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:39.524828 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:39.549223 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 459/910] 10.222.0.100 () {60 vars in 1146 bytes} [Tue Aug 2 08:58:39 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:58:39.549223 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217cda8-1881-42e0-a45a-2222437ad6f9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:39.603371 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670ae3d-af89-4102-a1cc-a3e0f70e33df None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:39.653898 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074d78f-d98a-4ba1-a7ef-efe32eb8321a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:39.654630 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074d78f-d98a-4ba1-a7ef-efe32eb8321a None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:39.654916 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074d78f-d98a-4ba1-a7ef-efe32eb8321a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:39.655195 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074d78f-d98a-4ba1-a7ef-efe32eb8321a None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:39.656001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074d78f-d98a-4ba1-a7ef-efe32eb8321a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:39.656313 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074d78f-d98a-4ba1-a7ef-efe32eb8321a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:39.657865 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074d78f-d98a-4ba1-a7ef-efe32eb8321a None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:39.681226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:39.689310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 452/911] 10.222.0.100 () {56 vars in 1154 bytes} [Tue Aug 2 08:58:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:39.692992 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074d78f-d98a-4ba1-a7ef-efe32eb8321a None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:39.708368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217cda8-1881-42e0-a45a-2222437ad6f9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:39.753586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299b98d-3f4f-48e9-b235-f0192ccfbed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:39.755468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299b98d-3f4f-48e9-b235-f0192ccfbed0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:39.755468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299b98d-3f4f-48e9-b235-f0192ccfbed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:39.755468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299b98d-3f4f-48e9-b235-f0192ccfbed0 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:39.755468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3299b98d-3f4f-48e9-b235-f0192ccfbed0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:39.756917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3299b98d-3f4f-48e9-b235-f0192ccfbed0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:39.763573 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 460/912] 10.222.0.100 () {56 vars in 1092 bytes} [Tue Aug 2 08:58:39 2022] GET /identity/v3/services => generated 1504 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 08:58:39.815976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:39.827115 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074d78f-d98a-4ba1-a7ef-efe32eb8321a None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:39.867359 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448015a-7436-450d-8e50-6b44aaa23fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:39.867985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448015a-7436-450d-8e50-6b44aaa23fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:39.868263 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448015a-7436-450d-8e50-6b44aaa23fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:39.868532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448015a-7436-450d-8e50-6b44aaa23fa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:39.944636 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448015a-7436-450d-8e50-6b44aaa23fa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:39.946127 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448015a-7436-450d-8e50-6b44aaa23fa0 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:40.041385 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:40.051943 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 453/913] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:58:39 2022] GET /identity/v3/auth/tokens => generated 1476 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:58:40.057981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299b98d-3f4f-48e9-b235-f0192ccfbed0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:40.167302 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299b98d-3f4f-48e9-b235-f0192ccfbed0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:40.204715 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b749531-84b3-459a-b91a-ff04dfc93b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:40.205449 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b749531-84b3-459a-b91a-ff04dfc93b51 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:40.205737 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b749531-84b3-459a-b91a-ff04dfc93b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:40.206074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b749531-84b3-459a-b91a-ff04dfc93b51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:40.217794 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b749531-84b3-459a-b91a-ff04dfc93b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:40.219242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b749531-84b3-459a-b91a-ff04dfc93b51 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:40.226311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:40.234970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-9b749531-84b3-459a-b91a-ff04dfc93b51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:40.234970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:40.234970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:40.234970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:40.234970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:40.234970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:40.234970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:40.234970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:40.234970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:40.234970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:40.234970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:40.234970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:40.234970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:40.234970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:40.234970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:40.235869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:40.235869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:40.235869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:40.235869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:40.235869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:40.235869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:40.235869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:40.235869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:40.235869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:40.235869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:40.235869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:40.235869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:40.235869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:40.235869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:40.235869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:40.235869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:40.235869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:40.237226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:40.237226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:40.237226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:40.237226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:40.237226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 461/914] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:40.242542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448015a-7436-450d-8e50-6b44aaa23fa0 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:40.348693 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448015a-7436-450d-8e50-6b44aaa23fa0 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:40.385263 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a790f-76ed-4f91-af50-287d2c585330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:40.386207 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a790f-76ed-4f91-af50-287d2c585330 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:40.386709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a790f-76ed-4f91-af50-287d2c585330 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:40.387177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a790f-76ed-4f91-af50-287d2c585330 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:40.388074 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a790f-76ed-4f91-af50-287d2c585330 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:40.388574 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a790f-76ed-4f91-af50-287d2c585330 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:40.390155 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a790f-76ed-4f91-af50-287d2c585330 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:40.402267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 454/915] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:40 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:40.407490 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b749531-84b3-459a-b91a-ff04dfc93b51 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:40.519532 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b749531-84b3-459a-b91a-ff04dfc93b51 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:40.557660 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3207e87-ffb9-464d-830c-e90cdea85b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:40.558314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3207e87-ffb9-464d-830c-e90cdea85b0c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:40.558598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3207e87-ffb9-464d-830c-e90cdea85b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:40.558873 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3207e87-ffb9-464d-830c-e90cdea85b0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:40.566027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3207e87-ffb9-464d-830c-e90cdea85b0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:40.567464 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3207e87-ffb9-464d-830c-e90cdea85b0c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:40.579766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-c3207e87-ffb9-464d-830c-e90cdea85b0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:40.579766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:40.579766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:40.579766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:40.579766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:40.579766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:40.579766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:40.579766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:40.579766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:40.579766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:40.579766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:40.579766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:40.579766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:40.579766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:40.579766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:40.580658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:40.580658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:40.580658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:40.580658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:40.580658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:40.580658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:40.580658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:40.580658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:40.580658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:40.580658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:40.580658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:40.580658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:40.580658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:40.580658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:40.580658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:40.580658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:40.580658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:40.581727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:40.581727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:40.581727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:40.581727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:40.581727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:40.581727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:40.581727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:40.581727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:40.581727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 462/916] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:40.587012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a790f-76ed-4f91-af50-287d2c585330 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:40.642840 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:40.691579 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a790f-76ed-4f91-af50-287d2c585330 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:40.726366 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb1645-d754-49b3-a1fc-7cab04be813c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:40.727320 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb1645-d754-49b3-a1fc-7cab04be813c None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:40.727825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb1645-d754-49b3-a1fc-7cab04be813c None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:40.728306 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb1645-d754-49b3-a1fc-7cab04be813c None admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:40.729287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb1645-d754-49b3-a1fc-7cab04be813c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:40.729790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb1645-d754-49b3-a1fc-7cab04be813c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:40.731516 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb1645-d754-49b3-a1fc-7cab04be813c None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:40.745193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 455/917] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:40.755694 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3207e87-ffb9-464d-830c-e90cdea85b0c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:40.822199 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:40.870654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3207e87-ffb9-464d-830c-e90cdea85b0c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:40.911254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a7adc-6a88-472e-94bd-7d71204a34f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:40.911884 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a7adc-6a88-472e-94bd-7d71204a34f2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:40.912470 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a7adc-6a88-472e-94bd-7d71204a34f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:40.913261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a7adc-6a88-472e-94bd-7d71204a34f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:40.915253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a7adc-6a88-472e-94bd-7d71204a34f2 None admin] RBAC: Adding query filter params (role.id=c3082fcfa7e74f6a91ca6cdedd44f0ac, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=1cd36b543bd9487ebd8ce578e9a76c78) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:40.915578 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a7adc-6a88-472e-94bd-7d71204a34f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:40.918935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a7adc-6a88-472e-94bd-7d71204a34f2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:40.923699 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 463/918] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:58:40 2022] GET /identity/v3/role_assignments?user.id=1cd36b543bd9487ebd8ce578e9a76c78&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=c3082fcfa7e74f6a91ca6cdedd44f0ac => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:41.108866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b18577-923b-477a-9891-003a063139b1 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:41.109376 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b18577-923b-477a-9891-003a063139b1 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:41.109376 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b18577-923b-477a-9891-003a063139b1 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:41.110976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 456/919] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:41.116231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99846e3-b120-490d-ad34-5266dfd505d1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:41.116520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99846e3-b120-490d-ad34-5266dfd505d1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:41.116801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99846e3-b120-490d-ad34-5266dfd505d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:41.136399 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b99846e3-b120-490d-ad34-5266dfd505d1 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:41.137610 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b99846e3-b120-490d-ad34-5266dfd505d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:41.207679 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 464/920] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:41 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:41.231488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b18577-923b-477a-9891-003a063139b1 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:41.344726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b18577-923b-477a-9891-003a063139b1 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:41.358735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:41.381423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6235b-bbb9-4631-8162-b7674c10f7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:41.382071 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6235b-bbb9-4631-8162-b7674c10f7e3 None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:41.382359 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6235b-bbb9-4631-8162-b7674c10f7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:41.382637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6235b-bbb9-4631-8162-b7674c10f7e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:41.383325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6235b-bbb9-4631-8162-b7674c10f7e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:41.384767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6235b-bbb9-4631-8162-b7674c10f7e3 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:41.387421 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: WARNING keystone.server.flask.common [None req-b8f6235b-bbb9-4631-8162-b7674c10f7e3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 02 08:58:41.406751 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.sql.core [None req-b8f6235b-bbb9-4631-8162-b7674c10f7e3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 02 08:58:41.406751 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 02 08:58:41.406751 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [parameters: {'id': '7228630ae398495b84528b9f5f87127f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 02 08:58:41.406751 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80361) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 02 08:58:41.415672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-b8f6235b-bbb9-4631-8162-b7674c10f7e3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7228630ae398495b84528b9f5f87127f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7228630ae398495b84528b9f5f87127f. Aug 02 08:58:41.415672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:41.415672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 08:58:41.415672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 08:58:41.415672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 08:58:41.415672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 08:58:41.415672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 08:58:41.415672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 08:58:41.415672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 08:58:41.415672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application conn.query(q) Aug 02 08:58:41.415672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 08:58:41.415672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 08:58:41.415672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 08:58:41.417532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application result.read() Aug 02 08:58:41.417532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 08:58:41.417532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 08:58:41.417532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 08:58:41.417532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 08:58:41.417532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 08:58:41.417532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 08:58:41.417532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 08:58:41.417532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 08:58:41.417532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 02 08:58:41.417532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:41.417532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 02 08:58:41.417532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:41.417532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:41.417532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 02 08:58:41.417532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 02 08:58:41.417532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 02 08:58:41.417532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 02 08:58:41.418790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 08:58:41.418790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application next(self.gen) Aug 02 08:58:41.418790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 02 08:58:41.418790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application yield resource Aug 02 08:58:41.418790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 08:58:41.418790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application next(self.gen) Aug 02 08:58:41.418790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 02 08:58:41.418790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self.session.rollback() Aug 02 08:58:41.418790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 02 08:58:41.418790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self.force_reraise() Aug 02 08:58:41.418790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 02 08:58:41.418790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise self.value Aug 02 08:58:41.418790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 02 08:58:41.418790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 02 08:58:41.418790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 02 08:58:41.418790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application session.commit() Aug 02 08:58:41.418790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 02 08:58:41.419935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 02 08:58:41.419935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 02 08:58:41.419935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._prepare_impl() Aug 02 08:58:41.419935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 02 08:58:41.419935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self.session.flush() Aug 02 08:58:41.419935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 02 08:58:41.419935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._flush(objects) Aug 02 08:58:41.419935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 02 08:58:41.419935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 02 08:58:41.419935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 02 08:58:41.419935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application compat.raise_( Aug 02 08:58:41.419935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 08:58:41.419935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception Aug 02 08:58:41.419935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 02 08:58:41.419935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application flush_context.execute() Aug 02 08:58:41.419935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 02 08:58:41.419935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rec.execute(self) Aug 02 08:58:41.421135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 02 08:58:41.421135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 02 08:58:41.421135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 02 08:58:41.421135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 02 08:58:41.421135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 02 08:58:41.421135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 02 08:58:41.421135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 02 08:58:41.421135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 02 08:58:41.421135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 02 08:58:41.421135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 02 08:58:41.421135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 02 08:58:41.421135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 02 08:58:41.421135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 02 08:58:41.421135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 02 08:58:41.421135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 02 08:58:41.422658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 02 08:58:41.422658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 08:58:41.422658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception Aug 02 08:58:41.422658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 08:58:41.422658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 08:58:41.422658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 08:58:41.422658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 08:58:41.422658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 08:58:41.422658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 08:58:41.422658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 08:58:41.422658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application conn.query(q) Aug 02 08:58:41.422658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 08:58:41.422658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 08:58:41.422658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 08:58:41.422658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application result.read() Aug 02 08:58:41.422658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 08:58:41.422658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 08:58:41.424823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 08:58:41.424823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 08:58:41.424823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 08:58:41.424823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 08:58:41.424823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 08:58:41.424823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 08:58:41.424823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 02 08:58:41.424823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 02 08:58:41.424823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [parameters: {'id': '7228630ae398495b84528b9f5f87127f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 02 08:58:41.424823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 02 08:58:41.424823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:41.424823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:41.424823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:41.424823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:41.424823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 02 08:58:41.434750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 02 08:58:41.434750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 02 08:58:41.434750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 02 08:58:41.434750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 02 08:58:41.434750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:41.434750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:41.434750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:41.434750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:41.434750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:41.434750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:41.434750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:41.434750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:41.434750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:41.434750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:41.434750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:41.434750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:41.434750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:41.436876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:41.436876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 02 08:58:41.436876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 02 08:58:41.436876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:41.436876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:41.436876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 02 08:58:41.436876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 02 08:58:41.436876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7228630ae398495b84528b9f5f87127f. Aug 02 08:58:41.436876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:41.436876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 457/921] 10.222.0.100 () {60 vars in 1145 bytes} [Tue Aug 2 08:58:41 2022] POST /identity/v3/projects => generated 262 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:41.438182 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99846e3-b120-490d-ad34-5266dfd505d1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:41.481529 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:41.533348 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99846e3-b120-490d-ad34-5266dfd505d1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:41.570648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dff63d-973d-452c-a4eb-cda14ce3ebbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:41.571300 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dff63d-973d-452c-a4eb-cda14ce3ebbd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:41.571585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dff63d-973d-452c-a4eb-cda14ce3ebbd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:41.571915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dff63d-973d-452c-a4eb-cda14ce3ebbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:41.579048 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dff63d-973d-452c-a4eb-cda14ce3ebbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:41.580597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dff63d-973d-452c-a4eb-cda14ce3ebbd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:41.593158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-21dff63d-973d-452c-a4eb-cda14ce3ebbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:41.593158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:41.593158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:41.593158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:41.593158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:41.593158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:41.593158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:41.593158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:41.593158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:41.593158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:41.593158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:41.593158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:41.593158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:41.593158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:41.593158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:41.595025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:41.595025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:41.595025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:41.595025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:41.595025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:41.595025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:41.595025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:41.595025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:41.595025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:41.595025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:41.595025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:41.595025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:41.595025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:41.595025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:41.595025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:41.595025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:41.595025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:41.596995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:41.596995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:41.596995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:41.596995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:41.596995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:41.596995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:41.596995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:41.596995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:41.596995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 465/922] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:41.601260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6235b-bbb9-4631-8162-b7674c10f7e3 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:41.711219 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6235b-bbb9-4631-8162-b7674c10f7e3 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:41.747357 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f37171-6bce-477b-848b-21ced8656ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:41.747981 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f37171-6bce-477b-848b-21ced8656ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:41.748263 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f37171-6bce-477b-848b-21ced8656ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:41.748540 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f37171-6bce-477b-848b-21ced8656ae5 None admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:41.749349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f37171-6bce-477b-848b-21ced8656ae5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:41.749930 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f37171-6bce-477b-848b-21ced8656ae5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:41.751711 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f37171-6bce-477b-848b-21ced8656ae5 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:41.773139 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 458/923] 10.222.0.100 () {56 vars in 1117 bytes} [Tue Aug 2 08:58:41 2022] GET /identity/v3/projects?name=service => generated 413 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:41.861273 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27596aa7-362e-426f-93a7-1a67405574c3 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:41.861607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27596aa7-362e-426f-93a7-1a67405574c3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:41.861828 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27596aa7-362e-426f-93a7-1a67405574c3 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:41.863488 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 466/924] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:41.869865 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea5504-236c-4d20-a57e-7c25eafdf2b8 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:41.870156 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea5504-236c-4d20-a57e-7c25eafdf2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:41.870420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea5504-236c-4d20-a57e-7c25eafdf2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:41.893251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-51ea5504-236c-4d20-a57e-7c25eafdf2b8 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:41.894878 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-51ea5504-236c-4d20-a57e-7c25eafdf2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:41.922190 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:41.946597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd65cd7-0b6c-4ccf-b560-a338c0f7b8ab None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:41.946905 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd65cd7-0b6c-4ccf-b560-a338c0f7b8ab None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:41.947141 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd65cd7-0b6c-4ccf-b560-a338c0f7b8ab None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:41.948763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 467/925] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:41.953776 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc902962-83cc-4bcb-b0ab-1fe1d1c38e7e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:41.954185 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc902962-83cc-4bcb-b0ab-1fe1d1c38e7e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:41.954464 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc902962-83cc-4bcb-b0ab-1fe1d1c38e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:41.973068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 459/926] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:41 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:41.975691 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-fc902962-83cc-4bcb-b0ab-1fe1d1c38e7e None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:41.976900 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fc902962-83cc-4bcb-b0ab-1fe1d1c38e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:42.003207 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea5504-236c-4d20-a57e-7c25eafdf2b8 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:42.051403 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 468/927] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:41 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:42.076474 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc902962-83cc-4bcb-b0ab-1fe1d1c38e7e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:42.125358 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea5504-236c-4d20-a57e-7c25eafdf2b8 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:42.128149 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:42.176448 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddad2a-9bbe-4e46-95f1-46ed1808d9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:42.177368 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddad2a-9bbe-4e46-95f1-46ed1808d9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:42.178442 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddad2a-9bbe-4e46-95f1-46ed1808d9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:42.179035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddad2a-9bbe-4e46-95f1-46ed1808d9a3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:42.179839 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ddad2a-9bbe-4e46-95f1-46ed1808d9a3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:42.181386 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ddad2a-9bbe-4e46-95f1-46ed1808d9a3 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:42.192710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-31ddad2a-9bbe-4e46-95f1-46ed1808d9a3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:58:42.192710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:42.192710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:42.192710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:42.192710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:42.192710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:42.192710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:42.192710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:42.192710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:58:42.192710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:42.192710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:42.192710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:42.192710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:42.192710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:42.192710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:42.192710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:42.192710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:42.194293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:42.194293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:42.194293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:42.194293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:42.194293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:42.194293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:42.194293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:42.194293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:42.194293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:42.194293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:42.194293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:42.194293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:42.194293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:42.194293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:42.194293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:42.194293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:42.195279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:42.195279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:42.195279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:42.195279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:42.195279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:42.195279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:42.195279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:42.195279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:42.195279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:42.195279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:42.195279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:42.195279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:42.195279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:58:42.195279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:42.197621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 460/928] 10.222.0.100 () {56 vars in 1131 bytes} [Tue Aug 2 08:58:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:42.203543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddad2a-9bbe-4e46-95f1-46ed1808d9a3 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:42.206079 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc902962-83cc-4bcb-b0ab-1fe1d1c38e7e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:42.230811 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:42.261676 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d3648-4bd8-47af-b0b1-6aeaa5152f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:42.262333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d3648-4bd8-47af-b0b1-6aeaa5152f4a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:42.262621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d3648-4bd8-47af-b0b1-6aeaa5152f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:42.262901 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d3648-4bd8-47af-b0b1-6aeaa5152f4a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:42.263541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d3648-4bd8-47af-b0b1-6aeaa5152f4a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:42.264901 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d3648-4bd8-47af-b0b1-6aeaa5152f4a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:42.279124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-313d3648-4bd8-47af-b0b1-6aeaa5152f4a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 08:58:42.279124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:42.279124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:42.279124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:42.279124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:42.279124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:42.279124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:42.279124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:42.279124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 08:58:42.279124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:42.279124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:42.279124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:42.279124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:42.279124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:42.279124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:42.279124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:42.280245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:42.280245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:42.280245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:42.280245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:42.280245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:42.280245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:42.280245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:42.280245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:42.280245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:42.280245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:42.280245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:42.280245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:42.280245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:42.280245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:42.280245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:42.280245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:42.281216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:42.281216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:42.281216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:42.281216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:42.281216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:42.281216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:42.281216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:42.281216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:42.281216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:42.281216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:42.281216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:42.281216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:42.281216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:42.281216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 08:58:42.281216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:42.281216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 469/929] 10.222.0.100 () {56 vars in 1137 bytes} [Tue Aug 2 08:58:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:42.286643 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d3648-4bd8-47af-b0b1-6aeaa5152f4a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:42.324224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddad2a-9bbe-4e46-95f1-46ed1808d9a3 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:42.363537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ade1e3-4a3a-425e-9328-16ced3dd9f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:42.364509 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ade1e3-4a3a-425e-9328-16ced3dd9f5f None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:42.365022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ade1e3-4a3a-425e-9328-16ced3dd9f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:42.365536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ade1e3-4a3a-425e-9328-16ced3dd9f5f None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:42.366575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ade1e3-4a3a-425e-9328-16ced3dd9f5f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:42.366879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ade1e3-4a3a-425e-9328-16ced3dd9f5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:42.368252 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ade1e3-4a3a-425e-9328-16ced3dd9f5f None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:42.373663 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 461/930] 10.222.0.100 () {56 vars in 1127 bytes} [Tue Aug 2 08:58:42 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:42.379531 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ade1e3-4a3a-425e-9328-16ced3dd9f5f None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:42.402294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d3648-4bd8-47af-b0b1-6aeaa5152f4a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:42.410032 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:42.441471 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14151fa3-5133-440b-a2cc-d6dc16cab15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:42.442204 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14151fa3-5133-440b-a2cc-d6dc16cab15f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:42.442501 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14151fa3-5133-440b-a2cc-d6dc16cab15f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:42.442879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14151fa3-5133-440b-a2cc-d6dc16cab15f None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:42.443799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14151fa3-5133-440b-a2cc-d6dc16cab15f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:42.444150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14151fa3-5133-440b-a2cc-d6dc16cab15f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:42.445772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14151fa3-5133-440b-a2cc-d6dc16cab15f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:42.455800 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 470/931] 10.222.0.100 () {56 vars in 1131 bytes} [Tue Aug 2 08:58:42 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:42.456815 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14151fa3-5133-440b-a2cc-d6dc16cab15f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:42.471341 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:42.487233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ade1e3-4a3a-425e-9328-16ced3dd9f5f None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:42.523675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90083e5-3598-4db8-adfe-4a614a3500d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:42.524286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90083e5-3598-4db8-adfe-4a614a3500d2 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:42.524598 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90083e5-3598-4db8-adfe-4a614a3500d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:42.524884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90083e5-3598-4db8-adfe-4a614a3500d2 None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:42.525588 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90083e5-3598-4db8-adfe-4a614a3500d2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:42.525890 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90083e5-3598-4db8-adfe-4a614a3500d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:42.527245 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90083e5-3598-4db8-adfe-4a614a3500d2 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:42.538082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 462/932] 10.222.0.100 () {56 vars in 1127 bytes} [Tue Aug 2 08:58:42 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:42.539457 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90083e5-3598-4db8-adfe-4a614a3500d2 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:42.569136 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14151fa3-5133-440b-a2cc-d6dc16cab15f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:42.607217 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764b5a9-db83-4dcc-a9b7-51b01bb0fbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:42.607856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764b5a9-db83-4dcc-a9b7-51b01bb0fbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:42.608145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764b5a9-db83-4dcc-a9b7-51b01bb0fbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:42.608428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764b5a9-db83-4dcc-a9b7-51b01bb0fbe7 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:42.609145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764b5a9-db83-4dcc-a9b7-51b01bb0fbe7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:42.609460 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764b5a9-db83-4dcc-a9b7-51b01bb0fbe7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:42.610954 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764b5a9-db83-4dcc-a9b7-51b01bb0fbe7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:42.621262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 471/933] 10.222.0.100 () {56 vars in 1131 bytes} [Tue Aug 2 08:58:42 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:42.622174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764b5a9-db83-4dcc-a9b7-51b01bb0fbe7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:42.648779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90083e5-3598-4db8-adfe-4a614a3500d2 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:42.674948 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:42.684984 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9bb63-2f52-449b-b00c-44c6573e9673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:42.685706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9bb63-2f52-449b-b00c-44c6573e9673 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:42.685986 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9bb63-2f52-449b-b00c-44c6573e9673 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:42.686256 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9bb63-2f52-449b-b00c-44c6573e9673 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:42.687007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9bb63-2f52-449b-b00c-44c6573e9673 None admin] RBAC: Adding query filter params (interface=public, service_id=bfedf2b10bee4c55b22422c796a8f147, region_id=RegionOne) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:42.687314 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9bb63-2f52-449b-b00c-44c6573e9673 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:42.688669 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9bb63-2f52-449b-b00c-44c6573e9673 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:42.699006 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 463/934] 10.222.0.100 () {56 vars in 1256 bytes} [Tue Aug 2 08:58:42 2022] GET /identity/v3/endpoints?service_id=bfedf2b10bee4c55b22422c796a8f147&interface=public®ion_id=RegionOne => generated 195 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:42.699882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9bb63-2f52-449b-b00c-44c6573e9673 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:42.716684 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:42.735601 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764b5a9-db83-4dcc-a9b7-51b01bb0fbe7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:42.774476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c071d19-4b37-4cf6-b63b-6271cf95f96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:42.775110 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c071d19-4b37-4cf6-b63b-6271cf95f96f None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:42.775395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c071d19-4b37-4cf6-b63b-6271cf95f96f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:42.775670 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c071d19-4b37-4cf6-b63b-6271cf95f96f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:42.776385 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c071d19-4b37-4cf6-b63b-6271cf95f96f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:42.777770 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c071d19-4b37-4cf6-b63b-6271cf95f96f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:42.808258 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9bb63-2f52-449b-b00c-44c6573e9673 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:42.809420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 472/935] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 08:58:42 2022] POST /identity/v3/endpoints => generated 349 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:58:42.855787 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0c946-405e-4359-8e23-46d0077bbeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:42.856484 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0c946-405e-4359-8e23-46d0077bbeb0 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:42.856768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0c946-405e-4359-8e23-46d0077bbeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:42.857042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0c946-405e-4359-8e23-46d0077bbeb0 None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:42.857811 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0c946-405e-4359-8e23-46d0077bbeb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:42.859293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0c946-405e-4359-8e23-46d0077bbeb0 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:42.866950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 464/936] 10.222.0.100 () {56 vars in 1092 bytes} [Tue Aug 2 08:58:42 2022] GET /identity/v3/services => generated 1504 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 08:58:43.088713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac9525-8a63-4ab5-80bf-bd17492df056 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:43.088900 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac9525-8a63-4ab5-80bf-bd17492df056 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:43.089216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac9525-8a63-4ab5-80bf-bd17492df056 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:43.090882 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 473/937] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:43.096137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcc107-9daa-4d1f-ba4a-4418a1eabc6e None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:43.096434 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcc107-9daa-4d1f-ba4a-4418a1eabc6e None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:43.096717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcc107-9daa-4d1f-ba4a-4418a1eabc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:43.116481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-c8bcc107-9daa-4d1f-ba4a-4418a1eabc6e None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:43.117762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c8bcc107-9daa-4d1f-ba4a-4418a1eabc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:43.190904 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 465/938] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:43 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:43.213837 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac9525-8a63-4ab5-80bf-bd17492df056 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:43.314966 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:43.326654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac9525-8a63-4ab5-80bf-bd17492df056 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:43.364834 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f25cc-9839-4fe9-8075-fde612f1623e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:43.365472 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f25cc-9839-4fe9-8075-fde612f1623e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:43.365759 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f25cc-9839-4fe9-8075-fde612f1623e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:43.366141 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f25cc-9839-4fe9-8075-fde612f1623e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:43.377600 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f25cc-9839-4fe9-8075-fde612f1623e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:43.383342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-6e6f25cc-9839-4fe9-8075-fde612f1623e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:58:43.383342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:43.383342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:43.383342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:43.383342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:43.383342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:43.383342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:43.383342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:43.383342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:43.383342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:43.383342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:43.383342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:43.383342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:43.383342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:43.383342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:43.383342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:43.384243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:43.384243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:43.384243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:43.384243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:43.384243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:43.384243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:43.384243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:43.384243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:43.384243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:43.384243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:43.384243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:43.384243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:43.384243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:43.384243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:43.384243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:43.384243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:43.385080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:43.385080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:43.385080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:43.385080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:43.385080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:43.385080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:43.385080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:43.385080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:43.385080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:43.385080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:43.385080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:58:43.385080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:43.385080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 474/939] 10.222.0.100 () {56 vars in 1125 bytes} [Tue Aug 2 08:58:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:43.390865 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bcc107-9daa-4d1f-ba4a-4418a1eabc6e None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:43.394309 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:43.499128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bcc107-9daa-4d1f-ba4a-4418a1eabc6e None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:43.538561 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70008d-a9af-4eed-8e28-ce5d831e2a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:43.539188 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70008d-a9af-4eed-8e28-ce5d831e2a9a None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:43.539473 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70008d-a9af-4eed-8e28-ce5d831e2a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:43.539740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70008d-a9af-4eed-8e28-ce5d831e2a9a None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:43.540487 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70008d-a9af-4eed-8e28-ce5d831e2a9a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:43.540792 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70008d-a9af-4eed-8e28-ce5d831e2a9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:43.542597 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70008d-a9af-4eed-8e28-ce5d831e2a9a None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:43.551758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 466/940] 10.222.0.100 () {56 vars in 1120 bytes} [Tue Aug 2 08:58:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:43.558122 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f25cc-9839-4fe9-8075-fde612f1623e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:43.670173 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f25cc-9839-4fe9-8075-fde612f1623e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:43.705753 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df4452-a95a-485c-9ea6-cef35f6eed86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:43.706399 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df4452-a95a-485c-9ea6-cef35f6eed86 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:43.706687 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df4452-a95a-485c-9ea6-cef35f6eed86 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:43.706969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df4452-a95a-485c-9ea6-cef35f6eed86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:43.718923 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df4452-a95a-485c-9ea6-cef35f6eed86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:43.720369 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df4452-a95a-485c-9ea6-cef35f6eed86 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:43.734791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-94df4452-a95a-485c-9ea6-cef35f6eed86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:43.734791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:43.734791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:43.734791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:43.734791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:43.734791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:43.734791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:43.734791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:43.734791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:43.734791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:43.734791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:43.734791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:43.734791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:43.734791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:43.734791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:43.736818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:43.736818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:43.736818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:43.736818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:43.736818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:43.736818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:43.736818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:43.736818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:43.736818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:43.736818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:43.736818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:43.736818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:43.736818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:43.736818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:43.736818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:43.736818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:43.736818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:43.738229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:43.738229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:43.738229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:43.738229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:43.738229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 475/941] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:43.742365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70008d-a9af-4eed-8e28-ce5d831e2a9a None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:43.819601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:43.858679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70008d-a9af-4eed-8e28-ce5d831e2a9a None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:43.894040 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f9cf4-3a8b-4baa-8dd3-0da3cf680c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:43.894661 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f9cf4-3a8b-4baa-8dd3-0da3cf680c82 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:43.894940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f9cf4-3a8b-4baa-8dd3-0da3cf680c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:43.895214 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f9cf4-3a8b-4baa-8dd3-0da3cf680c82 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:43.895987 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f9cf4-3a8b-4baa-8dd3-0da3cf680c82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:43.896291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f9cf4-3a8b-4baa-8dd3-0da3cf680c82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:43.897775 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f9cf4-3a8b-4baa-8dd3-0da3cf680c82 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:43.910339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 467/942] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:43.921329 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df4452-a95a-485c-9ea6-cef35f6eed86 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:43.924539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:44.010232 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ca7b3-9288-4222-8fab-f0adec6b9a0d None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:44.010574 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ca7b3-9288-4222-8fab-f0adec6b9a0d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:44.010802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ca7b3-9288-4222-8fab-f0adec6b9a0d None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:44.012294 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 468/943] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:44.018329 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1abcc-df0f-47af-8ade-019bd854b5a6 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:44.018686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1abcc-df0f-47af-8ade-019bd854b5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:44.018855 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1abcc-df0f-47af-8ade-019bd854b5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:44.037989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-b5b1abcc-df0f-47af-8ade-019bd854b5a6 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:44.038789 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df4452-a95a-485c-9ea6-cef35f6eed86 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:44.039198 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b5b1abcc-df0f-47af-8ade-019bd854b5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:44.086691 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788a866-fb4d-4a6e-8818-b99cc509b419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:44.087400 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788a866-fb4d-4a6e-8818-b99cc509b419 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:44.087678 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788a866-fb4d-4a6e-8818-b99cc509b419 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:44.087998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788a866-fb4d-4a6e-8818-b99cc509b419 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:44.099023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a788a866-fb4d-4a6e-8818-b99cc509b419 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:44.101854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a788a866-fb4d-4a6e-8818-b99cc509b419 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:44.117579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-a788a866-fb4d-4a6e-8818-b99cc509b419 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:58:44.117579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:44.117579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:44.117579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:44.117579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:44.117579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:44.117579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:44.117579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:44.117579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:44.117579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:44.117579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:44.117579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:44.117579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:44.117579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:44.117579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:44.117579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:44.119742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:44.119742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:44.119742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:44.119742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:44.119742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:44.119742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:44.119742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:44.119742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:44.119742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:44.119742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:44.119742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:44.119742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:44.119742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:44.119742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:44.119742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:44.119742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:44.119742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:44.119742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:44.121980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:44.121980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:44.121980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:44.121980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:44.121980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:44.121980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:44.121980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:44.121980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:44.121980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:58:44.121980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:44.123212 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 469/945] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:44 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:44.129168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 476/945] 10.222.0.100 () {56 vars in 1122 bytes} [Tue Aug 2 08:58:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:44.129382 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1abcc-df0f-47af-8ade-019bd854b5a6 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:44.149078 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788a866-fb4d-4a6e-8818-b99cc509b419 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:44.197853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:44.234425 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:44.256869 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1abcc-df0f-47af-8ade-019bd854b5a6 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:44.288292 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788a866-fb4d-4a6e-8818-b99cc509b419 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:44.301095 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2e7cb-a6d1-4852-a986-b5441b008409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:44.302690 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2e7cb-a6d1-4852-a986-b5441b008409 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:44.303201 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2e7cb-a6d1-4852-a986-b5441b008409 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:44.303680 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2e7cb-a6d1-4852-a986-b5441b008409 None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:44.305413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2e7cb-a6d1-4852-a986-b5441b008409 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:44.305748 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2e7cb-a6d1-4852-a986-b5441b008409 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:44.307200 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2e7cb-a6d1-4852-a986-b5441b008409 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:44.332553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 470/946] 10.222.0.100 () {56 vars in 1154 bytes} [Tue Aug 2 08:58:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:44.338898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724ab0f-43a2-4329-ae3e-c0bdb03e82c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:44.339544 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724ab0f-43a2-4329-ae3e-c0bdb03e82c3 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:44.339829 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724ab0f-43a2-4329-ae3e-c0bdb03e82c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:44.340110 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724ab0f-43a2-4329-ae3e-c0bdb03e82c3 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:44.340518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2e7cb-a6d1-4852-a986-b5441b008409 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:44.340851 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724ab0f-43a2-4329-ae3e-c0bdb03e82c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:44.342468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724ab0f-43a2-4329-ae3e-c0bdb03e82c3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:44.346199 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: WARNING keystone.server.flask.common [None req-5724ab0f-43a2-4329-ae3e-c0bdb03e82c3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 02 08:58:44.365224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.sql.core [None req-5724ab0f-43a2-4329-ae3e-c0bdb03e82c3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 02 08:58:44.365224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 02 08:58:44.365224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [parameters: {'user_id': '7f5a898af15546009724be390e7dba05', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 02 08:58:44.365224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80362) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 02 08:58:44.389896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-5724ab0f-43a2-4329-ae3e-c0bdb03e82c3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 02 08:58:44.389896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:44.389896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 08:58:44.389896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 08:58:44.389896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 08:58:44.389896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 08:58:44.389896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 08:58:44.389896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 08:58:44.389896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 08:58:44.389896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application conn.query(q) Aug 02 08:58:44.389896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 08:58:44.389896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 08:58:44.389896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 08:58:44.389896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application result.read() Aug 02 08:58:44.389896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 08:58:44.390904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 08:58:44.390904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 08:58:44.390904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 08:58:44.390904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 08:58:44.390904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 08:58:44.390904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 08:58:44.390904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 08:58:44.390904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 02 08:58:44.390904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:44.390904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 02 08:58:44.390904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:44.390904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:44.390904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 02 08:58:44.390904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 02 08:58:44.390904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 02 08:58:44.390904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 02 08:58:44.390904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 08:58:44.390904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application next(self.gen) Aug 02 08:58:44.391932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 02 08:58:44.391932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application yield resource Aug 02 08:58:44.391932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 08:58:44.391932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application next(self.gen) Aug 02 08:58:44.391932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 02 08:58:44.391932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self.session.rollback() Aug 02 08:58:44.391932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 02 08:58:44.391932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self.force_reraise() Aug 02 08:58:44.391932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 02 08:58:44.391932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise self.value Aug 02 08:58:44.391932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 02 08:58:44.391932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 02 08:58:44.391932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 02 08:58:44.391932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application session.commit() Aug 02 08:58:44.391932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 02 08:58:44.391932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 02 08:58:44.391932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 02 08:58:44.392849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._prepare_impl() Aug 02 08:58:44.392849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 02 08:58:44.392849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self.session.flush() Aug 02 08:58:44.392849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 02 08:58:44.392849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._flush(objects) Aug 02 08:58:44.392849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 02 08:58:44.392849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 02 08:58:44.392849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 02 08:58:44.392849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application compat.raise_( Aug 02 08:58:44.392849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 08:58:44.392849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception Aug 02 08:58:44.392849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 02 08:58:44.392849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application flush_context.execute() Aug 02 08:58:44.392849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 02 08:58:44.392849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rec.execute(self) Aug 02 08:58:44.392849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 02 08:58:44.392849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 02 08:58:44.400769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 02 08:58:44.400769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 02 08:58:44.400769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 02 08:58:44.400769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 02 08:58:44.400769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 02 08:58:44.400769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 02 08:58:44.400769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 02 08:58:44.400769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 02 08:58:44.400769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 02 08:58:44.400769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 02 08:58:44.400769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 02 08:58:44.400769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 02 08:58:44.400769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 02 08:58:44.400769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 02 08:58:44.400769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 08:58:44.400769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception Aug 02 08:58:44.403365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 08:58:44.403365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 08:58:44.403365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 08:58:44.403365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 08:58:44.403365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 08:58:44.403365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 08:58:44.403365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 08:58:44.403365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application conn.query(q) Aug 02 08:58:44.403365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 08:58:44.403365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 08:58:44.403365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 08:58:44.403365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application result.read() Aug 02 08:58:44.403365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 08:58:44.403365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 08:58:44.403365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 08:58:44.403365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 08:58:44.406233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 08:58:44.406233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 08:58:44.406233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 08:58:44.406233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 08:58:44.406233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 02 08:58:44.406233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 02 08:58:44.406233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [parameters: {'user_id': '7f5a898af15546009724be390e7dba05', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 02 08:58:44.406233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 02 08:58:44.406233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:44.406233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:44.406233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:44.406233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:44.406233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:44.406233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:44.406233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:44.408410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:44.408410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:44.408410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:44.408410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:44.408410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:44.408410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:44.408410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:44.408410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 02 08:58:44.408410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 02 08:58:44.408410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:44.408410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:44.408410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:44.408410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:44.408410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:44.408410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:44.408410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 02 08:58:44.408410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 02 08:58:44.411845 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 02 08:58:44.411845 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 02 08:58:44.411845 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 02 08:58:44.411845 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 02 08:58:44.411845 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 02 08:58:44.411845 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:44.411845 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 477/947] 10.222.0.100 () {60 vars in 1136 bytes} [Tue Aug 2 08:58:44 2022] POST /identity/v3/users => generated 160 bytes in 246 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:44.411845 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724ab0f-43a2-4329-ae3e-c0bdb03e82c3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:44.473575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2e7cb-a6d1-4852-a986-b5441b008409 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:44.489770 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:44.513810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99611392-a4ef-4b24-b254-fde4dc2d4d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:44.514819 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99611392-a4ef-4b24-b254-fde4dc2d4d58 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:44.515302 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99611392-a4ef-4b24-b254-fde4dc2d4d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:44.515798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99611392-a4ef-4b24-b254-fde4dc2d4d58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:44.517303 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724ab0f-43a2-4329-ae3e-c0bdb03e82c3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:44.528202 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99611392-a4ef-4b24-b254-fde4dc2d4d58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:44.530042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99611392-a4ef-4b24-b254-fde4dc2d4d58 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:44.543528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:44.550507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-99611392-a4ef-4b24-b254-fde4dc2d4d58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:44.550507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:44.550507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:44.550507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:44.550507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:44.550507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:44.550507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:44.550507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:44.550507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:44.550507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:44.550507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:44.550507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:44.550507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:44.550507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:44.550507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:44.557222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:44.557222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:44.557222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:44.557222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:44.557222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:44.557222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:44.557222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:44.557222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:44.557222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:44.557222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:44.557222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:44.557222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:44.557222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:44.557222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:44.557222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:44.557222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:44.557222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:44.558170 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:44.558170 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:44.558170 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:44.558170 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:44.558170 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 471/948] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:44.561589 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99611392-a4ef-4b24-b254-fde4dc2d4d58 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:44.567428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5166faa-f9f6-472e-82ff-f926da826f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:44.568093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5166faa-f9f6-472e-82ff-f926da826f65 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:44.568419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5166faa-f9f6-472e-82ff-f926da826f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:44.568695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5166faa-f9f6-472e-82ff-f926da826f65 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:44.577193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5166faa-f9f6-472e-82ff-f926da826f65 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:44.578813 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5166faa-f9f6-472e-82ff-f926da826f65 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:44.591291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-a5166faa-f9f6-472e-82ff-f926da826f65 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 08:58:44.591291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:44.591291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:44.591291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:44.591291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:44.591291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:44.591291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:44.591291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:44.591291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:44.591291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:44.591291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:44.591291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:44.591291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:44.591291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:44.591291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:44.591291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:44.592357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:44.592357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:44.592357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:44.592357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:44.592357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:44.592357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:44.592357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:44.592357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:44.592357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:44.592357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:44.592357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:44.592357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:44.592357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:44.592357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:44.592357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:44.592357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:44.592357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:44.592357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:44.593956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:44.593956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:44.593956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:44.593956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:44.593956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:44.593956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:44.593956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:44.593956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:44.593956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 08:58:44.593956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:44.597267 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 478/949] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:58:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:44.598108 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5166faa-f9f6-472e-82ff-f926da826f65 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:44.681502 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99611392-a4ef-4b24-b254-fde4dc2d4d58 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:44.707569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5166faa-f9f6-472e-82ff-f926da826f65 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:44.719059 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034e881-da94-4834-8cdc-98ae109be636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:44.719985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034e881-da94-4834-8cdc-98ae109be636 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:44.720492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034e881-da94-4834-8cdc-98ae109be636 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:44.720971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034e881-da94-4834-8cdc-98ae109be636 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:44.721922 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034e881-da94-4834-8cdc-98ae109be636 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:44.722431 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034e881-da94-4834-8cdc-98ae109be636 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:44.724129 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034e881-da94-4834-8cdc-98ae109be636 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:44.737249 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 472/950] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:44.743588 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034e881-da94-4834-8cdc-98ae109be636 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:44.753030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd67fa3-8a4d-433b-8262-e1b3a8e715c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:44.753822 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd67fa3-8a4d-433b-8262-e1b3a8e715c3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:44.754105 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd67fa3-8a4d-433b-8262-e1b3a8e715c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:44.754381 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd67fa3-8a4d-433b-8262-e1b3a8e715c3 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:44.755145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd67fa3-8a4d-433b-8262-e1b3a8e715c3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:44.755449 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd67fa3-8a4d-433b-8262-e1b3a8e715c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:44.757654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd67fa3-8a4d-433b-8262-e1b3a8e715c3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:44.778213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:44.805621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 479/951] 10.222.0.100 () {56 vars in 1106 bytes} [Tue Aug 2 08:58:44 2022] GET /identity/v3/users?name=cinder => generated 356 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:44.864059 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034e881-da94-4834-8cdc-98ae109be636 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:44.911431 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c81d37-addf-41ef-8b81-6988b926af3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:44.912699 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c81d37-addf-41ef-8b81-6988b926af3e None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:44.912984 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c81d37-addf-41ef-8b81-6988b926af3e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:44.913298 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c81d37-addf-41ef-8b81-6988b926af3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:44.930941 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c81d37-addf-41ef-8b81-6988b926af3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:44.932791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c81d37-addf-41ef-8b81-6988b926af3e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:44.946781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-04c81d37-addf-41ef-8b81-6988b926af3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:44.946781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:44.946781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:44.946781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:44.946781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:44.946781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:44.946781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:44.946781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:44.946781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:44.946781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:44.946781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:44.946781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:44.946781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:44.946781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:44.946781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:44.947836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:44.947836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:44.947836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:44.947836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:44.947836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:44.947836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:44.947836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:44.947836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:44.947836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:44.947836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:44.947836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:44.947836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:44.947836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:44.947836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:44.947836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:44.947836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:44.947836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:44.948756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:44.948756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:44.948756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:44.948756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:44.948756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:44.948756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:44.948756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:44.948756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:44.951497 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 473/952] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:44.961150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd67fa3-8a4d-433b-8262-e1b3a8e715c3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:44.967129 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af85679-7784-434c-b839-a8a33358e5b9 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:44.967129 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af85679-7784-434c-b839-a8a33358e5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:44.967375 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af85679-7784-434c-b839-a8a33358e5b9 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:44.968870 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 474/953] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:44.974207 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2dccac-6dd4-4e29-8281-b8e6b40f0c88 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:44.974993 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2dccac-6dd4-4e29-8281-b8e6b40f0c88 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:44.975272 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2dccac-6dd4-4e29-8281-b8e6b40f0c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:44.993721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-ec2dccac-6dd4-4e29-8281-b8e6b40f0c88 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:44.994911 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ec2dccac-6dd4-4e29-8281-b8e6b40f0c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:45.012905 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:45.076335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 475/954] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:44 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:58:45.077803 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb393b00-f813-48a4-8a5e-949315ce463d None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:45.078086 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb393b00-f813-48a4-8a5e-949315ce463d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:45.078365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb393b00-f813-48a4-8a5e-949315ce463d None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:45.079659 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 476/955] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:45.086919 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf68f54-d192-4faa-bb97-1f858c150180 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:45.087208 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf68f54-d192-4faa-bb97-1f858c150180 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:45.087499 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf68f54-d192-4faa-bb97-1f858c150180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:45.089693 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd67fa3-8a4d-433b-8262-e1b3a8e715c3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:45.105494 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-8bf68f54-d192-4faa-bb97-1f858c150180 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:45.106677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-8bf68f54-d192-4faa-bb97-1f858c150180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:45.117597 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:45.125029 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953660d-3f92-4284-b1eb-cfee0767a318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:45.125684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953660d-3f92-4284-b1eb-cfee0767a318 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:45.125968 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953660d-3f92-4284-b1eb-cfee0767a318 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:45.126242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953660d-3f92-4284-b1eb-cfee0767a318 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:45.126989 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953660d-3f92-4284-b1eb-cfee0767a318 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:45.127298 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953660d-3f92-4284-b1eb-cfee0767a318 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:45.128713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953660d-3f92-4284-b1eb-cfee0767a318 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:45.141401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 480/956] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:45.142186 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953660d-3f92-4284-b1eb-cfee0767a318 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:45.178731 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 477/957] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:45 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:45.179552 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf68f54-d192-4faa-bb97-1f858c150180 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:45.247506 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953660d-3f92-4284-b1eb-cfee0767a318 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:45.248942 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:45.278942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf68f54-d192-4faa-bb97-1f858c150180 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:45.281344 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fad722-a5f4-4168-9822-a56da644d708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:45.281984 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fad722-a5f4-4168-9822-a56da644d708 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:45.282263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fad722-a5f4-4168-9822-a56da644d708 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:45.282536 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fad722-a5f4-4168-9822-a56da644d708 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:45.283156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fad722-a5f4-4168-9822-a56da644d708 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:45.284490 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fad722-a5f4-4168-9822-a56da644d708 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:45.293340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-b3fad722-a5f4-4168-9822-a56da644d708 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 08:58:45.293340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:45.293340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:45.293340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:45.293340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:45.293340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:45.293340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:45.293340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:45.293340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 08:58:45.293340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:45.293340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:45.293340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:45.293340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:45.293340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:45.293340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:45.293340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:45.293340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:45.299821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:45.299821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:45.299821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:45.299821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:45.299821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:45.299821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:45.299821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:45.299821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:45.299821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:45.299821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:45.299821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:45.299821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:45.299821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:45.299821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:45.299821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:45.299821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:45.300940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:45.300940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:45.300940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:45.300940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:45.300940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:45.300940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:45.300940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:45.300940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:45.300940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:45.300940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:45.300940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:45.300940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:45.300940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 08:58:45.300940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:45.300940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 481/958] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:45.301918 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fad722-a5f4-4168-9822-a56da644d708 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:45.312788 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3ed28-d6c5-42f8-9a9f-bb662b245839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:45.313476 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3ed28-d6c5-42f8-9a9f-bb662b245839 None admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:45.313754 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3ed28-d6c5-42f8-9a9f-bb662b245839 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:45.314026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3ed28-d6c5-42f8-9a9f-bb662b245839 None admin] PATH_INFO: `/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/1cd36b543bd9487ebd8ce578e9a76c78/roles/c3082fcfa7e74f6a91ca6cdedd44f0ac` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:45.316187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff3ed28-d6c5-42f8-9a9f-bb662b245839 None admin] RBAC: Authorizing `identity:create_grant(project_id=61ca0176324e445b94e42a5214d2c89b, user_id=1cd36b543bd9487ebd8ce578e9a76c78, role_id=c3082fcfa7e74f6a91ca6cdedd44f0ac)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:45.317640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff3ed28-d6c5-42f8-9a9f-bb662b245839 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:45.330902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 478/959] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:58:45 2022] PUT /identity/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/1cd36b543bd9487ebd8ce578e9a76c78/roles/c3082fcfa7e74f6a91ca6cdedd44f0ac => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:58:45.338175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3ed28-d6c5-42f8-9a9f-bb662b245839 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:45.347792 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:45.418451 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fad722-a5f4-4168-9822-a56da644d708 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:45.457850 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0cc347-f86d-4b27-866d-705d3eec9bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:45.458128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3ed28-d6c5-42f8-9a9f-bb662b245839 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:45.458517 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0cc347-f86d-4b27-866d-705d3eec9bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:45.459332 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0cc347-f86d-4b27-866d-705d3eec9bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:45.459614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0cc347-f86d-4b27-866d-705d3eec9bd0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:45.460251 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0cc347-f86d-4b27-866d-705d3eec9bd0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:45.461697 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0cc347-f86d-4b27-866d-705d3eec9bd0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:45.471598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-db0cc347-f86d-4b27-866d-705d3eec9bd0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:58:45.471598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:45.471598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:45.471598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:45.471598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:45.471598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:45.471598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:45.471598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:45.471598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:58:45.471598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:45.471598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:45.471598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:45.471598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:45.471598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:45.471598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:45.471598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:45.471598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:45.472911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:45.472911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:45.472911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:45.472911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:45.472911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:45.472911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:45.472911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:45.472911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:45.472911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:45.472911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:45.472911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:45.472911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:45.472911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:45.472911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:45.472911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:45.472911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:45.473976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:45.473976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:45.473976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:45.473976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:45.473976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:45.473976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:45.473976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:45.473976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:45.473976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:45.473976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:45.473976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:45.473976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:45.473976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:58:45.473976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:45.473976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 482/960] 10.222.0.100 () {56 vars in 1131 bytes} [Tue Aug 2 08:58:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:45.480303 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0cc347-f86d-4b27-866d-705d3eec9bd0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:45.511200 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb288e0d-c1ed-4c66-bddc-518eee161b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:45.512100 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb288e0d-c1ed-4c66-bddc-518eee161b58 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:45.512389 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb288e0d-c1ed-4c66-bddc-518eee161b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:45.512669 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb288e0d-c1ed-4c66-bddc-518eee161b58 None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:45.513409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb288e0d-c1ed-4c66-bddc-518eee161b58 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:45.513871 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb288e0d-c1ed-4c66-bddc-518eee161b58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:45.515845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb288e0d-c1ed-4c66-bddc-518eee161b58 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:45.516904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:45.527921 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 479/961] 10.222.0.100 () {56 vars in 1117 bytes} [Tue Aug 2 08:58:45 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:45.529419 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb288e0d-c1ed-4c66-bddc-518eee161b58 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:45.596218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0cc347-f86d-4b27-866d-705d3eec9bd0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:45.605008 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:45.634416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9ed96-db64-4009-9ed9-89558fd41f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:45.635048 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9ed96-db64-4009-9ed9-89558fd41f1c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:45.635340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9ed96-db64-4009-9ed9-89558fd41f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:45.635619 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9ed96-db64-4009-9ed9-89558fd41f1c None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:45.636305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9ed96-db64-4009-9ed9-89558fd41f1c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:45.636619 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9ed96-db64-4009-9ed9-89558fd41f1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:45.637995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9ed96-db64-4009-9ed9-89558fd41f1c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:45.643171 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 483/962] 10.222.0.100 () {56 vars in 1127 bytes} [Tue Aug 2 08:58:45 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:45.643461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb288e0d-c1ed-4c66-bddc-518eee161b58 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:45.648746 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9ed96-db64-4009-9ed9-89558fd41f1c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:45.684012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da63fb6-66cb-4844-bdea-36f81d49507f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:45.684637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da63fb6-66cb-4844-bdea-36f81d49507f None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:45.684921 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da63fb6-66cb-4844-bdea-36f81d49507f None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:45.685230 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da63fb6-66cb-4844-bdea-36f81d49507f None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:45.685971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da63fb6-66cb-4844-bdea-36f81d49507f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:45.686283 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da63fb6-66cb-4844-bdea-36f81d49507f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:45.687698 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da63fb6-66cb-4844-bdea-36f81d49507f None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:45.701158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 480/963] 10.222.0.100 () {56 vars in 1117 bytes} [Tue Aug 2 08:58:45 2022] GET /identity/v3/services?type=compute => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:45.762562 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:45.771115 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9ed96-db64-4009-9ed9-89558fd41f1c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:45.809922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18e20a-b3c0-4791-9b41-9569176eafb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:45.810719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18e20a-b3c0-4791-9b41-9569176eafb5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:45.811109 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18e20a-b3c0-4791-9b41-9569176eafb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:45.811489 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18e20a-b3c0-4791-9b41-9569176eafb5 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:45.812290 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18e20a-b3c0-4791-9b41-9569176eafb5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:45.812704 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18e20a-b3c0-4791-9b41-9569176eafb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:45.814263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18e20a-b3c0-4791-9b41-9569176eafb5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:45.824959 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 484/964] 10.222.0.100 () {56 vars in 1127 bytes} [Tue Aug 2 08:58:45 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:45.826474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da63fb6-66cb-4844-bdea-36f81d49507f None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:45.945030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da63fb6-66cb-4844-bdea-36f81d49507f None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:45.980579 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173caa79-d0f9-489e-9c81-d80ce9b68123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:45.981294 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173caa79-d0f9-489e-9c81-d80ce9b68123 None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:45.981582 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173caa79-d0f9-489e-9c81-d80ce9b68123 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:45.981855 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173caa79-d0f9-489e-9c81-d80ce9b68123 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:45.982557 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173caa79-d0f9-489e-9c81-d80ce9b68123 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:45.983913 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173caa79-d0f9-489e-9c81-d80ce9b68123 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:45.988441 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:46.013219 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 481/965] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 08:58:45 2022] POST /identity/v3/endpoints => generated 351 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:58:47.336907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8219c-9458-4026-8f5b-d837603f9325 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:47.337457 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8219c-9458-4026-8f5b-d837603f9325 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:47.337570 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8219c-9458-4026-8f5b-d837603f9325 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:47.338992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 485/966] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:47.343925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb89c62-00a9-4a63-aca8-5a5bd7218707 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:47.344358 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb89c62-00a9-4a63-aca8-5a5bd7218707 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:47.344501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb89c62-00a9-4a63-aca8-5a5bd7218707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:47.362631 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-0eb89c62-00a9-4a63-aca8-5a5bd7218707 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:47.363819 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-0eb89c62-00a9-4a63-aca8-5a5bd7218707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:47.439124 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 482/967] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:47 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:47.463272 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8219c-9458-4026-8f5b-d837603f9325 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:47.567918 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8219c-9458-4026-8f5b-d837603f9325 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:47.602047 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e654af1-2bf4-4277-9afd-ae3e527840eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:47.602616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e654af1-2bf4-4277-9afd-ae3e527840eb None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:47.602901 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e654af1-2bf4-4277-9afd-ae3e527840eb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:47.603177 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e654af1-2bf4-4277-9afd-ae3e527840eb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:47.614593 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e654af1-2bf4-4277-9afd-ae3e527840eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:47.620125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-9e654af1-2bf4-4277-9afd-ae3e527840eb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:58:47.620125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:47.620125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:47.620125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:47.620125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:47.620125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:47.620125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:47.620125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:47.620125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:47.620125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:47.620125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:47.620125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:47.620125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:47.620125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:47.620125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:47.620125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:47.621174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:47.621174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:47.621174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:47.621174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:47.621174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:47.621174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:47.621174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:47.621174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:47.621174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:47.621174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:47.621174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:47.621174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:47.621174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:47.621174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:47.621174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:47.621174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:47.622066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:47.622066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:47.622066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:47.622066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:47.622066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:47.622066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:47.622066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:47.622066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:47.622066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:47.622066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:47.622066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:58:47.622066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:47.625433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 486/968] 10.222.0.100 () {56 vars in 1125 bytes} [Tue Aug 2 08:58:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:47.627429 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb89c62-00a9-4a63-aca8-5a5bd7218707 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:47.704641 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8d135-33e7-4536-9547-83458d0b5e20 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:47.704983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8d135-33e7-4536-9547-83458d0b5e20 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:47.705224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8d135-33e7-4536-9547-83458d0b5e20 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:47.706724 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 487/969] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:58:47.711778 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc49f51-620a-495a-87d3-930933ab2f13 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:47.712047 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc49f51-620a-495a-87d3-930933ab2f13 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:47.712326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc49f51-620a-495a-87d3-930933ab2f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:47.729449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb89c62-00a9-4a63-aca8-5a5bd7218707 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:47.730695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-afc49f51-620a-495a-87d3-930933ab2f13 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:47.732313 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-afc49f51-620a-495a-87d3-930933ab2f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:47.746472 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:47.747459 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:47.768556 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6abe42-19fe-4859-ae96-2de2113e7f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:47.769310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6abe42-19fe-4859-ae96-2de2113e7f7a None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:47.769594 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6abe42-19fe-4859-ae96-2de2113e7f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:47.769872 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6abe42-19fe-4859-ae96-2de2113e7f7a None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:47.770640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6abe42-19fe-4859-ae96-2de2113e7f7a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:47.770890 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6abe42-19fe-4859-ae96-2de2113e7f7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:47.772458 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6abe42-19fe-4859-ae96-2de2113e7f7a None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:47.785045 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 483/970] 10.222.0.100 () {56 vars in 1120 bytes} [Tue Aug 2 08:58:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:47.786140 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6abe42-19fe-4859-ae96-2de2113e7f7a None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:47.810433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 488/971] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:47 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:58:47.832524 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc49f51-620a-495a-87d3-930933ab2f13 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:47.895850 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6abe42-19fe-4859-ae96-2de2113e7f7a None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:47.937165 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc49f51-620a-495a-87d3-930933ab2f13 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:47.938829 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8ed5e-db1a-44f4-a7f0-8913ab7d3a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:47.939587 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8ed5e-db1a-44f4-a7f0-8913ab7d3a11 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:47.939867 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8ed5e-db1a-44f4-a7f0-8913ab7d3a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:47.940139 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8ed5e-db1a-44f4-a7f0-8913ab7d3a11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:47.951699 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8ed5e-db1a-44f4-a7f0-8913ab7d3a11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:47.953989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8ed5e-db1a-44f4-a7f0-8913ab7d3a11 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:47.972347 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff1197-3ae6-45e8-871c-9e1672efd5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:47.972513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-d3f8ed5e-db1a-44f4-a7f0-8913ab7d3a11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:47.972513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:47.972513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:47.972513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:47.972513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:47.972513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:47.972513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:47.972513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:47.972513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:47.972513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:47.972513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:47.972513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:47.972513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:47.972513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:47.972513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:47.973650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:47.973650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:47.973650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:47.973650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:47.973650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:47.973650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:47.973650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:47.973650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:47.973650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:47.973650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:47.973650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:47.973650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:47.973650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:47.973650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:47.973650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:47.973650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:47.973650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:47.975128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:47.975128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:47.975128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:47.975128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:47.975355 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff1197-3ae6-45e8-871c-9e1672efd5b9 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:47.975355 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff1197-3ae6-45e8-871c-9e1672efd5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:47.975355 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff1197-3ae6-45e8-871c-9e1672efd5b9 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:47.975355 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff1197-3ae6-45e8-871c-9e1672efd5b9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:47.976097 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff1197-3ae6-45e8-871c-9e1672efd5b9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:47.979080 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 484/972] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:47.980062 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8ed5e-db1a-44f4-a7f0-8913ab7d3a11 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:47.993160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 489/973] 10.222.0.100 () {60 vars in 1146 bytes} [Tue Aug 2 08:58:47 2022] POST /identity/v3/services => generated 243 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 08:58:48.024703 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:48.033914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd06309-1b24-4923-928a-118f12277bca None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:48.034233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd06309-1b24-4923-928a-118f12277bca None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:48.034474 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd06309-1b24-4923-928a-118f12277bca None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:48.039105 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 490/974] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:58:48.042199 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869d3c1-4e42-411d-ba80-8766a54f54f7 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:48.042525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869d3c1-4e42-411d-ba80-8766a54f54f7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:48.042757 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869d3c1-4e42-411d-ba80-8766a54f54f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:48.066896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8869d3c1-4e42-411d-ba80-8766a54f54f7 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:48.069904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8869d3c1-4e42-411d-ba80-8766a54f54f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:48.072644 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:48.107854 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8ed5e-db1a-44f4-a7f0-8913ab7d3a11 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:48.156009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e80b16-d70e-481d-9f4d-19dffe21bca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:48.157146 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e80b16-d70e-481d-9f4d-19dffe21bca0 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:48.157590 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e80b16-d70e-481d-9f4d-19dffe21bca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:48.157877 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e80b16-d70e-481d-9f4d-19dffe21bca0 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:48.159552 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e80b16-d70e-481d-9f4d-19dffe21bca0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:48.159552 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e80b16-d70e-481d-9f4d-19dffe21bca0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:48.159746 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 491/975] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:48 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:48.160441 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e80b16-d70e-481d-9f4d-19dffe21bca0 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:48.176348 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 485/976] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:48.181981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869d3c1-4e42-411d-ba80-8766a54f54f7 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:48.182539 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e80b16-d70e-481d-9f4d-19dffe21bca0 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:48.281381 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:48.290888 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869d3c1-4e42-411d-ba80-8766a54f54f7 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:48.299087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e80b16-d70e-481d-9f4d-19dffe21bca0 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:48.327417 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea351b5c-4e78-4a4e-a392-9a5a4c327347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:48.328086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea351b5c-4e78-4a4e-a392-9a5a4c327347 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:48.328376 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea351b5c-4e78-4a4e-a392-9a5a4c327347 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:48.328656 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea351b5c-4e78-4a4e-a392-9a5a4c327347 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:48.329330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea351b5c-4e78-4a4e-a392-9a5a4c327347 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:48.330690 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea351b5c-4e78-4a4e-a392-9a5a4c327347 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:48.332706 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:48.340455 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705ab3a-1814-4f45-aa4d-9849e8025e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:48.341168 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705ab3a-1814-4f45-aa4d-9849e8025e68 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:48.341456 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705ab3a-1814-4f45-aa4d-9849e8025e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:48.341729 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705ab3a-1814-4f45-aa4d-9849e8025e68 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:48.341846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ea351b5c-4e78-4a4e-a392-9a5a4c327347 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:58:48.341846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:48.341846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:48.341846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:48.341846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:48.341846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:48.341846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:48.341846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:48.341846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:58:48.341846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:48.341846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:48.341846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:48.341846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:48.341846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:48.341846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:48.341846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:48.341846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:48.343126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:48.343126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:48.343126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:48.343126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:48.343126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:48.343126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:48.343126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:48.343126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:48.343126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:48.343126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:48.343126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:48.343126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:48.343126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:48.343126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:48.343126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:48.343126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:48.343975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:48.343975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:48.343975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:48.343975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:48.343975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:48.343975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:48.343975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:48.343975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:48.343975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:48.343975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:48.343975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:48.343975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:48.343975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:58:48.343975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:48.343975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 492/977] 10.222.0.100 () {56 vars in 1131 bytes} [Tue Aug 2 08:58:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:48.348696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea351b5c-4e78-4a4e-a392-9a5a4c327347 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:48.352631 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705ab3a-1814-4f45-aa4d-9849e8025e68 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:48.354117 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705ab3a-1814-4f45-aa4d-9849e8025e68 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:48.367098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-1705ab3a-1814-4f45-aa4d-9849e8025e68 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:58:48.367098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:48.367098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:48.367098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:48.367098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:48.367098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:48.367098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:48.367098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:48.367098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:48.367098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:48.367098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:48.367098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:48.367098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:48.367098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:48.367098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:48.367098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:48.368073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:48.368073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:48.368073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:48.368073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:48.368073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:48.368073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:48.368073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:48.368073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:48.368073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:48.368073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:48.368073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:48.368073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:48.368073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:48.368073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:48.368073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:48.368073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:48.368073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:48.368073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:48.369012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:48.369012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:48.369012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:48.369012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:48.369012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:48.369012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:48.369012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:48.369012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:48.369012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:58:48.369012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:48.369012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 486/978] 10.222.0.100 () {56 vars in 1122 bytes} [Tue Aug 2 08:58:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:48.374011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705ab3a-1814-4f45-aa4d-9849e8025e68 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:48.458939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea351b5c-4e78-4a4e-a392-9a5a4c327347 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:48.488139 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705ab3a-1814-4f45-aa4d-9849e8025e68 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:48.497940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd2e3d-fc9e-40f8-93af-0bb2fe5715f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:48.498650 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd2e3d-fc9e-40f8-93af-0bb2fe5715f0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:48.499008 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd2e3d-fc9e-40f8-93af-0bb2fe5715f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:48.499710 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd2e3d-fc9e-40f8-93af-0bb2fe5715f0 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:48.500467 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd2e3d-fc9e-40f8-93af-0bb2fe5715f0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:48.500866 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd2e3d-fc9e-40f8-93af-0bb2fe5715f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:48.502456 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd2e3d-fc9e-40f8-93af-0bb2fe5715f0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:48.508066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 493/979] 10.222.0.100 () {56 vars in 1127 bytes} [Tue Aug 2 08:58:48 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:48.514025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd2e3d-fc9e-40f8-93af-0bb2fe5715f0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:48.524498 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974d3bc-c52b-4dcd-835e-961cb069e8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:48.525158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974d3bc-c52b-4dcd-835e-961cb069e8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:48.525544 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974d3bc-c52b-4dcd-835e-961cb069e8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:48.525822 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974d3bc-c52b-4dcd-835e-961cb069e8b3 None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:48.526615 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974d3bc-c52b-4dcd-835e-961cb069e8b3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:48.527052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974d3bc-c52b-4dcd-835e-961cb069e8b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:48.528479 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974d3bc-c52b-4dcd-835e-961cb069e8b3 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:48.549306 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 487/980] 10.222.0.100 () {56 vars in 1154 bytes} [Tue Aug 2 08:58:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:48.555709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974d3bc-c52b-4dcd-835e-961cb069e8b3 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:48.561497 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:48.590959 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:48.638135 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd2e3d-fc9e-40f8-93af-0bb2fe5715f0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:48.665598 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974d3bc-c52b-4dcd-835e-961cb069e8b3 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:48.674445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bbc05-c725-46eb-b1fe-7fd31de9835d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:48.675105 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bbc05-c725-46eb-b1fe-7fd31de9835d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:48.675387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bbc05-c725-46eb-b1fe-7fd31de9835d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:48.675656 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bbc05-c725-46eb-b1fe-7fd31de9835d None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:48.676322 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4bbc05-c725-46eb-b1fe-7fd31de9835d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:48.676628 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4bbc05-c725-46eb-b1fe-7fd31de9835d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:48.678028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4bbc05-c725-46eb-b1fe-7fd31de9835d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:48.683359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 494/981] 10.222.0.100 () {56 vars in 1127 bytes} [Tue Aug 2 08:58:48 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:48.689091 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bbc05-c725-46eb-b1fe-7fd31de9835d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:48.706762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e1d78-04a8-4295-b8a1-40e02937c541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:48.707483 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e1d78-04a8-4295-b8a1-40e02937c541 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:48.707774 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e1d78-04a8-4295-b8a1-40e02937c541 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:48.708049 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e1d78-04a8-4295-b8a1-40e02937c541 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:48.781141 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e1d78-04a8-4295-b8a1-40e02937c541 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:48.782639 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e1d78-04a8-4295-b8a1-40e02937c541 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:48.786330 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:48.797730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bbc05-c725-46eb-b1fe-7fd31de9835d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:48.824624 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:48.835736 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17825825-3e00-4333-a232-b6fca06f7acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:48.836500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17825825-3e00-4333-a232-b6fca06f7acc None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:48.836881 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17825825-3e00-4333-a232-b6fca06f7acc None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:48.837424 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17825825-3e00-4333-a232-b6fca06f7acc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:48.838129 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17825825-3e00-4333-a232-b6fca06f7acc None admin] RBAC: Adding query filter params (interface=admin, service_id=bfedf2b10bee4c55b22422c796a8f147, region_id=RegionOne) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:48.838510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17825825-3e00-4333-a232-b6fca06f7acc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:48.839989 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17825825-3e00-4333-a232-b6fca06f7acc None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:48.850499 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 495/982] 10.222.0.100 () {56 vars in 1254 bytes} [Tue Aug 2 08:58:48 2022] GET /identity/v3/endpoints?service_id=bfedf2b10bee4c55b22422c796a8f147&interface=admin®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:48.852621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17825825-3e00-4333-a232-b6fca06f7acc None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:48.896802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 488/983] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:58:48 2022] GET /identity/v3/auth/tokens => generated 1572 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 08:58:48.902200 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e1d78-04a8-4295-b8a1-40e02937c541 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:48.972357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17825825-3e00-4333-a232-b6fca06f7acc None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:49.011335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e1d78-04a8-4295-b8a1-40e02937c541 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:49.014687 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:49.015903 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a70198-ad04-48e4-a3b5-b62292b3e029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:49.016556 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a70198-ad04-48e4-a3b5-b62292b3e029 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:49.016846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a70198-ad04-48e4-a3b5-b62292b3e029 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:49.017126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a70198-ad04-48e4-a3b5-b62292b3e029 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:49.017795 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a70198-ad04-48e4-a3b5-b62292b3e029 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:49.019154 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a70198-ad04-48e4-a3b5-b62292b3e029 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:49.026660 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 496/984] 10.222.0.100 () {56 vars in 1092 bytes} [Tue Aug 2 08:58:48 2022] GET /identity/v3/services => generated 1735 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 08:58:49.053544 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb10555-6d4a-4cbd-a54c-3984e2217018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:49.054105 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb10555-6d4a-4cbd-a54c-3984e2217018 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:49.054387 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb10555-6d4a-4cbd-a54c-3984e2217018 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:49.054661 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb10555-6d4a-4cbd-a54c-3984e2217018 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:49.066445 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb10555-6d4a-4cbd-a54c-3984e2217018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:49.068058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb10555-6d4a-4cbd-a54c-3984e2217018 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:49.086679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-6cb10555-6d4a-4cbd-a54c-3984e2217018 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:49.086679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:49.086679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:49.086679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:49.086679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:49.086679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:49.086679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:49.086679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:49.086679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:49.086679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:49.086679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:49.086679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:49.086679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:49.086679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:49.086679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:49.087675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:49.087675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:49.087675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:49.087675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:49.087675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:49.087675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:49.087675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:49.087675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:49.087675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:49.087675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:49.087675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:49.087675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:49.087675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:49.087675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:49.087675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:49.087675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:49.087675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:49.088591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:49.088591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:49.088591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:49.088591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:49.088591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 489/985] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:49.093905 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a70198-ad04-48e4-a3b5-b62292b3e029 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:49.156696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:49.207784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a70198-ad04-48e4-a3b5-b62292b3e029 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:49.243226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d413d2-2195-4ee1-857b-6ad32b8ff468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:49.243851 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d413d2-2195-4ee1-857b-6ad32b8ff468 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:49.244139 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d413d2-2195-4ee1-857b-6ad32b8ff468 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:49.244415 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d413d2-2195-4ee1-857b-6ad32b8ff468 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:49.245104 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d413d2-2195-4ee1-857b-6ad32b8ff468 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:49.245441 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d413d2-2195-4ee1-857b-6ad32b8ff468 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:49.246790 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d413d2-2195-4ee1-857b-6ad32b8ff468 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:49.259091 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 497/986] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:49.264560 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb10555-6d4a-4cbd-a54c-3984e2217018 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:49.365172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb10555-6d4a-4cbd-a54c-3984e2217018 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:49.399544 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc26b73-d3f8-454d-b594-c759c216e516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:49.400184 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc26b73-d3f8-454d-b594-c759c216e516 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:49.400470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc26b73-d3f8-454d-b594-c759c216e516 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:49.400745 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc26b73-d3f8-454d-b594-c759c216e516 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:49.407473 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc26b73-d3f8-454d-b594-c759c216e516 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:49.408919 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc26b73-d3f8-454d-b594-c759c216e516 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:49.420360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-8bc26b73-d3f8-454d-b594-c759c216e516 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:49.420360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:49.420360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:49.420360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:49.420360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:49.420360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:49.420360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:49.420360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:49.420360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:49.420360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:49.420360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:49.420360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:49.420360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:49.420360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:49.420360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:49.421324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:49.421324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:49.421324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:49.421324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:49.421324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:49.421324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:49.421324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:49.421324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:49.421324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:49.421324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:49.421324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:49.421324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:49.421324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:49.421324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:49.421324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:49.421324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:49.421324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:49.422241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:49.422241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:49.422241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:49.422241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:49.422241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:49.422241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:49.422241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:49.422241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:49.425194 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 490/987] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:49.427563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d413d2-2195-4ee1-857b-6ad32b8ff468 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:49.529897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d413d2-2195-4ee1-857b-6ad32b8ff468 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:49.544273 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:49.566212 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c8b12-9405-426d-8dea-4186709da0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:49.566927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c8b12-9405-426d-8dea-4186709da0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:49.567332 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c8b12-9405-426d-8dea-4186709da0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:49.567679 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c8b12-9405-426d-8dea-4186709da0e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:49.569073 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c8b12-9405-426d-8dea-4186709da0e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:49.569502 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c8b12-9405-426d-8dea-4186709da0e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:49.570982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c8b12-9405-426d-8dea-4186709da0e1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:49.588558 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 498/988] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:49.589880 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc26b73-d3f8-454d-b594-c759c216e516 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:49.608863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:49.694798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc26b73-d3f8-454d-b594-c759c216e516 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:49.728862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb8d50-bcbf-4196-985c-52c7afc5aa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:49.729654 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb8d50-bcbf-4196-985c-52c7afc5aa36 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:49.730034 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb8d50-bcbf-4196-985c-52c7afc5aa36 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:49.730415 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb8d50-bcbf-4196-985c-52c7afc5aa36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:49.731302 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb8d50-bcbf-4196-985c-52c7afc5aa36 None admin] RBAC: Adding query filter params (role.id=c3082fcfa7e74f6a91ca6cdedd44f0ac, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=1cd36b543bd9487ebd8ce578e9a76c78) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:49.731736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb8d50-bcbf-4196-985c-52c7afc5aa36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:49.733311 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb8d50-bcbf-4196-985c-52c7afc5aa36 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:49.745214 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 491/989] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:58:49 2022] GET /identity/v3/role_assignments?user.id=1cd36b543bd9487ebd8ce578e9a76c78&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=c3082fcfa7e74f6a91ca6cdedd44f0ac => generated 610 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:50.065965 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e668bc-b713-44ad-b1d6-86c6cc20425f None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:50.066315 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e668bc-b713-44ad-b1d6-86c6cc20425f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:50.066536 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e668bc-b713-44ad-b1d6-86c6cc20425f None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:50.068625 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 499/990] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:50.073700 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823c78d-03ea-485a-a6fc-0d11ab9b59e8 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:50.073996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823c78d-03ea-485a-a6fc-0d11ab9b59e8 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:50.074279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823c78d-03ea-485a-a6fc-0d11ab9b59e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:50.092949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-a823c78d-03ea-485a-a6fc-0d11ab9b59e8 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:50.094169 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-a823c78d-03ea-485a-a6fc-0d11ab9b59e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:50.163334 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 492/991] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:50 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:50.185905 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e668bc-b713-44ad-b1d6-86c6cc20425f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:50.293437 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e668bc-b713-44ad-b1d6-86c6cc20425f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:50.327126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300cf08-e40d-429b-a3e6-10628a525eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:50.327699 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300cf08-e40d-429b-a3e6-10628a525eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:50.327979 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300cf08-e40d-429b-a3e6-10628a525eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:50.328253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300cf08-e40d-429b-a3e6-10628a525eb2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:50.328874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300cf08-e40d-429b-a3e6-10628a525eb2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:50.330280 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300cf08-e40d-429b-a3e6-10628a525eb2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:50.340023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-7300cf08-e40d-429b-a3e6-10628a525eb2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 08:58:50.340023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:50.340023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:50.340023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:50.340023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:50.340023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:50.340023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:50.340023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:50.340023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 08:58:50.340023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:50.340023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:50.340023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:50.340023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:50.340023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:50.340023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:50.340023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:50.340023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:50.341030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:50.341030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:50.341030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:50.341030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:50.341030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:50.341030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:50.341030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:50.341030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:50.341030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:50.341030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:50.341030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:50.341030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:50.341030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:50.341030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:50.341030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:50.341030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:50.342454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:50.342454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:50.342454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:50.342454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:50.342454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:50.342454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:50.342454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:50.342454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:50.342454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:50.342454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:50.342454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:50.342454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:50.342454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 08:58:50.342454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:50.345453 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 500/992] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:50.347022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823c78d-03ea-485a-a6fc-0d11ab9b59e8 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:50.359545 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:50.450312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823c78d-03ea-485a-a6fc-0d11ab9b59e8 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:50.484356 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b9d98-39df-44f4-9d90-9bfcaa74fad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:50.484976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b9d98-39df-44f4-9d90-9bfcaa74fad9 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:50.485295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b9d98-39df-44f4-9d90-9bfcaa74fad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:50.485577 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b9d98-39df-44f4-9d90-9bfcaa74fad9 None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:50.486257 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b9d98-39df-44f4-9d90-9bfcaa74fad9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:50.486566 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b9d98-39df-44f4-9d90-9bfcaa74fad9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:50.487935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b9d98-39df-44f4-9d90-9bfcaa74fad9 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:50.494721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 493/993] 10.222.0.100 () {56 vars in 1117 bytes} [Tue Aug 2 08:58:50 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:50.500519 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300cf08-e40d-429b-a3e6-10628a525eb2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:50.542362 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:50.607691 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300cf08-e40d-429b-a3e6-10628a525eb2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:50.641460 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed9c40-d6d4-423f-9f9f-f4b4e6bff6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:50.642075 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed9c40-d6d4-423f-9f9f-f4b4e6bff6fd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:50.642353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed9c40-d6d4-423f-9f9f-f4b4e6bff6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:50.642629 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed9c40-d6d4-423f-9f9f-f4b4e6bff6fd None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:50.643296 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed9c40-d6d4-423f-9f9f-f4b4e6bff6fd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:50.643599 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed9c40-d6d4-423f-9f9f-f4b4e6bff6fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:50.644942 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed9c40-d6d4-423f-9f9f-f4b4e6bff6fd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:50.655437 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 501/994] 10.222.0.100 () {56 vars in 1117 bytes} [Tue Aug 2 08:58:50 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:50.656763 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b9d98-39df-44f4-9d90-9bfcaa74fad9 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:50.737091 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:50.763957 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b9d98-39df-44f4-9d90-9bfcaa74fad9 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:50.798715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cefea7-0171-4743-812c-0b458609e376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:50.799328 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cefea7-0171-4743-812c-0b458609e376 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:50.799608 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cefea7-0171-4743-812c-0b458609e376 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:50.799879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cefea7-0171-4743-812c-0b458609e376 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:50.800622 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cefea7-0171-4743-812c-0b458609e376 None admin] RBAC: Adding query filter params (interface=public, service_id=35997a1213dc4eb29427c2f00e07668b, region_id=RegionOne) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:50.800929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cefea7-0171-4743-812c-0b458609e376 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:50.804285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cefea7-0171-4743-812c-0b458609e376 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:50.813336 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 494/995] 10.222.0.100 () {56 vars in 1256 bytes} [Tue Aug 2 08:58:50 2022] GET /identity/v3/endpoints?service_id=35997a1213dc4eb29427c2f00e07668b&interface=public®ion_id=RegionOne => generated 195 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:50.815394 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed9c40-d6d4-423f-9f9f-f4b4e6bff6fd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:50.931581 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed9c40-d6d4-423f-9f9f-f4b4e6bff6fd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:50.933187 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:50.967041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b794ea3-8090-4587-8498-4a925f518bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:50.967669 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b794ea3-8090-4587-8498-4a925f518bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:50.967951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b794ea3-8090-4587-8498-4a925f518bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:50.968227 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b794ea3-8090-4587-8498-4a925f518bb8 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:50.968882 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b794ea3-8090-4587-8498-4a925f518bb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:50.970279 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b794ea3-8090-4587-8498-4a925f518bb8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:50.977426 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 502/996] 10.222.0.100 () {56 vars in 1092 bytes} [Tue Aug 2 08:58:50 2022] GET /identity/v3/services => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 08:58:51.119719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a2be7a-9d28-45a4-b833-23f8ded193eb None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:51.119992 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a2be7a-9d28-45a4-b833-23f8ded193eb None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:51.120271 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a2be7a-9d28-45a4-b833-23f8ded193eb None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:51.121672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 495/997] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:51.127398 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b097d8-ad18-4806-b977-4401de77803e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:51.127693 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b097d8-ad18-4806-b977-4401de77803e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:51.127974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b097d8-ad18-4806-b977-4401de77803e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:51.147668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-05b097d8-ad18-4806-b977-4401de77803e None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:51.148877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-05b097d8-ad18-4806-b977-4401de77803e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:51.216629 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 503/998] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:51 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:58:51.237965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a2be7a-9d28-45a4-b833-23f8ded193eb None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:51.343144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a2be7a-9d28-45a4-b833-23f8ded193eb None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:51.377063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e73ceb-52d0-4307-89c0-be5c48791df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:51.377726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e73ceb-52d0-4307-89c0-be5c48791df7 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:51.378006 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e73ceb-52d0-4307-89c0-be5c48791df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:51.378283 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e73ceb-52d0-4307-89c0-be5c48791df7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:51.378903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e73ceb-52d0-4307-89c0-be5c48791df7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:51.380867 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e73ceb-52d0-4307-89c0-be5c48791df7 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:51.389794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-f6e73ceb-52d0-4307-89c0-be5c48791df7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:58:51.389794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:51.389794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:51.389794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:51.389794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:51.389794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:51.389794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:51.389794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:51.389794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:58:51.389794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:51.389794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:51.389794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:51.389794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:51.389794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:51.389794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:51.389794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:51.389794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:51.390821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:51.390821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:51.390821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:51.390821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:51.390821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:51.390821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:51.390821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:51.390821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:51.390821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:51.390821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:51.390821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:51.390821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:51.390821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:51.390821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:51.390821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:51.390821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:51.391670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:51.391670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:51.391670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:51.391670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:51.391670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:51.391670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:51.391670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:51.391670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:51.391670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:51.391670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:51.391670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:51.391670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:51.391670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:58:51.391670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:51.393243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 496/999] 10.222.0.100 () {56 vars in 1131 bytes} [Tue Aug 2 08:58:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:51.397570 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b097d8-ad18-4806-b977-4401de77803e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:51.493483 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:51.501168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b097d8-ad18-4806-b977-4401de77803e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:51.535723 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d801d1-c57f-4066-ac87-a0acf66fe1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:51.536337 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d801d1-c57f-4066-ac87-a0acf66fe1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:51.537209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d801d1-c57f-4066-ac87-a0acf66fe1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:51.537485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d801d1-c57f-4066-ac87-a0acf66fe1b5 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:51.538249 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d801d1-c57f-4066-ac87-a0acf66fe1b5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:51.538557 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d801d1-c57f-4066-ac87-a0acf66fe1b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:51.539918 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d801d1-c57f-4066-ac87-a0acf66fe1b5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:51.549613 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 504/1000] 10.222.0.100 () {56 vars in 1127 bytes} [Tue Aug 2 08:58:51 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:51.551396 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e73ceb-52d0-4307-89c0-be5c48791df7 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:51.554652 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:51.657072 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e73ceb-52d0-4307-89c0-be5c48791df7 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:51.692574 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32309d74-758d-4be1-8140-7cd5126e835b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:51.693364 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32309d74-758d-4be1-8140-7cd5126e835b None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:51.693664 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32309d74-758d-4be1-8140-7cd5126e835b None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:51.693979 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32309d74-758d-4be1-8140-7cd5126e835b None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:51.694649 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32309d74-758d-4be1-8140-7cd5126e835b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:51.694951 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32309d74-758d-4be1-8140-7cd5126e835b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:51.696298 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32309d74-758d-4be1-8140-7cd5126e835b None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:51.701433 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 497/1001] 10.222.0.100 () {56 vars in 1127 bytes} [Tue Aug 2 08:58:51 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:51.707003 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d801d1-c57f-4066-ac87-a0acf66fe1b5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:51.810472 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d801d1-c57f-4066-ac87-a0acf66fe1b5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:51.811099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf45180-de02-405d-9836-3cea2c98189f None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:51.811391 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf45180-de02-405d-9836-3cea2c98189f None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:51.811668 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf45180-de02-405d-9836-3cea2c98189f None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:51.813013 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 498/1002] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:58:51.817786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca1ca2-f326-45bd-8698-74d0d3f1a6c9 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:51.818074 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca1ca2-f326-45bd-8698-74d0d3f1a6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:51.818320 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca1ca2-f326-45bd-8698-74d0d3f1a6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:51.835812 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-28ca1ca2-f326-45bd-8698-74d0d3f1a6c9 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:51.836985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-28ca1ca2-f326-45bd-8698-74d0d3f1a6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:51.845858 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61beebac-3e96-4ede-9af0-a8ed3971aae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:51.846475 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61beebac-3e96-4ede-9af0-a8ed3971aae7 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:51.846756 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61beebac-3e96-4ede-9af0-a8ed3971aae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:51.847030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61beebac-3e96-4ede-9af0-a8ed3971aae7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:51.847740 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61beebac-3e96-4ede-9af0-a8ed3971aae7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:51.849647 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61beebac-3e96-4ede-9af0-a8ed3971aae7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:51.867200 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 505/1003] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 08:58:51 2022] POST /identity/v3/endpoints => generated 327 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:58:51.922912 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:51.941159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 499/1004] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:51 2022] POST /identity/v3/auth/tokens => generated 1720 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:58:51.959601 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61beebac-3e96-4ede-9af0-a8ed3971aae7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:51.983648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:52.072431 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61beebac-3e96-4ede-9af0-a8ed3971aae7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:52.110111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8a660-2cdd-49e2-b009-1d456303e267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:52.110766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8a660-2cdd-49e2-b009-1d456303e267 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:52.111054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8a660-2cdd-49e2-b009-1d456303e267 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:52.111326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8a660-2cdd-49e2-b009-1d456303e267 None admin] PATH_INFO: `/v3/services/image` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:52.111955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8a660-2cdd-49e2-b009-1d456303e267 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:52.113395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8a660-2cdd-49e2-b009-1d456303e267 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:52.122716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-35c8a660-2cdd-49e2-b009-1d456303e267 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 08:58:52.122716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:52.122716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:52.122716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:52.122716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:52.122716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:52.122716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:52.122716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:52.122716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 08:58:52.122716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:52.122716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:52.122716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:52.122716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:52.122716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:52.122716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:52.122716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:52.122716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:52.123733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:52.123733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:52.123733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:52.123733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:52.123733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:52.123733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:52.123733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:52.123733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:52.123733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:52.123733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:52.123733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:52.123733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:52.123733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:52.123733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:52.123733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:52.123733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:52.124857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:52.124857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:52.124857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:52.124857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:52.124857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:52.124857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:52.124857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:52.124857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:52.124857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:52.124857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:52.124857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:52.124857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:52.124857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 08:58:52.124857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:52.124857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 506/1005] 10.222.0.100 () {56 vars in 1110 bytes} [Tue Aug 2 08:58:51 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:52.130099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca1ca2-f326-45bd-8698-74d0d3f1a6c9 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:52.231947 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca1ca2-f326-45bd-8698-74d0d3f1a6c9 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:52.266810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c8def-7fbb-4c7f-b37e-6a6015128491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:52.267477 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c8def-7fbb-4c7f-b37e-6a6015128491 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:52.267766 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c8def-7fbb-4c7f-b37e-6a6015128491 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:52.268042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c8def-7fbb-4c7f-b37e-6a6015128491 None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:52.268716 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05c8def-7fbb-4c7f-b37e-6a6015128491 None admin] RBAC: Adding query filter params (name=image) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:52.269021 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05c8def-7fbb-4c7f-b37e-6a6015128491 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:52.270424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05c8def-7fbb-4c7f-b37e-6a6015128491 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:52.275833 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 500/1006] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:52 2022] GET /identity/v3/services?name=image => generated 123 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:52.281374 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8a660-2cdd-49e2-b009-1d456303e267 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:52.370851 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:52.388916 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8a660-2cdd-49e2-b009-1d456303e267 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:52.423051 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677caed7-699a-4f07-a432-fdf8d7f69178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:52.423675 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677caed7-699a-4f07-a432-fdf8d7f69178 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:52.424023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677caed7-699a-4f07-a432-fdf8d7f69178 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:52.424304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677caed7-699a-4f07-a432-fdf8d7f69178 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:52.424978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677caed7-699a-4f07-a432-fdf8d7f69178 None admin] RBAC: Adding query filter params (type=image) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:52.425313 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677caed7-699a-4f07-a432-fdf8d7f69178 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:52.426657 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677caed7-699a-4f07-a432-fdf8d7f69178 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:52.437158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 507/1007] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:52 2022] GET /identity/v3/services?type=image => generated 123 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:52.990390 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9b54d-5a66-4448-a0f6-0bbecbe0046f None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:52.990895 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9b54d-5a66-4448-a0f6-0bbecbe0046f None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:52.991003 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9b54d-5a66-4448-a0f6-0bbecbe0046f None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:52.992423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 501/1008] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:52 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:52.997321 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893b024-b3e1-48e1-88b7-f6f20f17efe3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:52.997714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893b024-b3e1-48e1-88b7-f6f20f17efe3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:52.997884 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893b024-b3e1-48e1-88b7-f6f20f17efe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:53.017434 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9893b024-b3e1-48e1-88b7-f6f20f17efe3 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:53.018661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9893b024-b3e1-48e1-88b7-f6f20f17efe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:53.087039 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 508/1009] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:52 2022] POST /identity/v3/auth/tokens => generated 1720 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:53.108583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9b54d-5a66-4448-a0f6-0bbecbe0046f None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:53.166692 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:53.211792 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9b54d-5a66-4448-a0f6-0bbecbe0046f None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:53.245441 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc65b04-555e-4279-9bc3-0c29d3f4fd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:53.246120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc65b04-555e-4279-9bc3-0c29d3f4fd5d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:53.246398 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc65b04-555e-4279-9bc3-0c29d3f4fd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:53.246669 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc65b04-555e-4279-9bc3-0c29d3f4fd5d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:53.247275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc65b04-555e-4279-9bc3-0c29d3f4fd5d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:53.248632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc65b04-555e-4279-9bc3-0c29d3f4fd5d None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:53.257802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-bcc65b04-555e-4279-9bc3-0c29d3f4fd5d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 08:58:53.257802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:53.257802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:53.257802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:53.257802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:53.257802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:53.257802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:53.257802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:53.257802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 08:58:53.257802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:53.257802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:53.257802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:53.257802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:53.257802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:53.257802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:53.257802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:53.257802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:53.258785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:53.258785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:53.258785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:53.258785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:53.258785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:53.258785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:53.258785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:53.258785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:53.258785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:53.258785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:53.258785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:53.258785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:53.258785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:53.258785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:53.258785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:53.258785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:53.259671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:53.259671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:53.259671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:53.259671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:53.259671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:53.259671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:53.259671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:53.259671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:53.259671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:53.259671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:53.259671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:53.259671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:53.259671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 08:58:53.259671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:53.261164 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 502/1010] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:53.264797 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893b024-b3e1-48e1-88b7-f6f20f17efe3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:53.342051 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:53.364768 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893b024-b3e1-48e1-88b7-f6f20f17efe3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:53.399232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c245f0d-306d-4ae9-95ab-0ff26e935b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:53.399850 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c245f0d-306d-4ae9-95ab-0ff26e935b39 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:53.400129 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c245f0d-306d-4ae9-95ab-0ff26e935b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:53.400401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c245f0d-306d-4ae9-95ab-0ff26e935b39 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:53.401072 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c245f0d-306d-4ae9-95ab-0ff26e935b39 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:53.401401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c245f0d-306d-4ae9-95ab-0ff26e935b39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:53.402744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c245f0d-306d-4ae9-95ab-0ff26e935b39 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:53.412346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 509/1011] 10.222.0.100 () {56 vars in 1117 bytes} [Tue Aug 2 08:58:53 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:53.413297 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc65b04-555e-4279-9bc3-0c29d3f4fd5d None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:53.513858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc65b04-555e-4279-9bc3-0c29d3f4fd5d None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:53.534359 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:53.551503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa752e-5767-4aa4-b8ff-3d9f14ef56d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:53.552766 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa752e-5767-4aa4-b8ff-3d9f14ef56d4 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:53.553158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa752e-5767-4aa4-b8ff-3d9f14ef56d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:53.553492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa752e-5767-4aa4-b8ff-3d9f14ef56d4 None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:53.554529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa752e-5767-4aa4-b8ff-3d9f14ef56d4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:53.554838 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa752e-5767-4aa4-b8ff-3d9f14ef56d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:53.556343 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa752e-5767-4aa4-b8ff-3d9f14ef56d4 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:53.561666 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 503/1012] 10.222.0.100 () {56 vars in 1117 bytes} [Tue Aug 2 08:58:53 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:53.567044 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c245f0d-306d-4ae9-95ab-0ff26e935b39 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:53.669064 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c245f0d-306d-4ae9-95ab-0ff26e935b39 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:53.702448 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3c89b-70f4-4742-8280-a4a3f910b222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:53.703065 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3c89b-70f4-4742-8280-a4a3f910b222 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:53.703350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3c89b-70f4-4742-8280-a4a3f910b222 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:53.703625 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3c89b-70f4-4742-8280-a4a3f910b222 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:53.704335 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3c89b-70f4-4742-8280-a4a3f910b222 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:53.705713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3c89b-70f4-4742-8280-a4a3f910b222 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:53.720399 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:53.729445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 510/1013] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 08:58:53 2022] POST /identity/v3/endpoints => generated 336 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:58:53.925478 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2474a7-45df-4885-91ac-a241c7791441 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:53.925776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2474a7-45df-4885-91ac-a241c7791441 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:53.926057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2474a7-45df-4885-91ac-a241c7791441 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:53.927617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 504/1014] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:53.932921 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b0b7b-a47c-4e97-829d-e846f71f4213 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:53.933290 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b0b7b-a47c-4e97-829d-e846f71f4213 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:53.933577 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b0b7b-a47c-4e97-829d-e846f71f4213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:53.952755 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-fe6b0b7b-a47c-4e97-829d-e846f71f4213 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:53.953959 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fe6b0b7b-a47c-4e97-829d-e846f71f4213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:54.033150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 511/1015] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:53 2022] POST /identity/v3/auth/tokens => generated 1877 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:54.056274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2474a7-45df-4885-91ac-a241c7791441 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:54.160520 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2474a7-45df-4885-91ac-a241c7791441 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:54.195661 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e9677-ae52-492a-97ea-06597dc1d605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:54.196230 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e9677-ae52-492a-97ea-06597dc1d605 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:54.196516 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e9677-ae52-492a-97ea-06597dc1d605 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:54.196795 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e9677-ae52-492a-97ea-06597dc1d605 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:54.197963 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e9677-ae52-492a-97ea-06597dc1d605 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:54.199403 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e9677-ae52-492a-97ea-06597dc1d605 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:54.201174 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 505/1016] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:54 2022] GET /identity/v3/domains/default => generated 200 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:54.207445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b0b7b-a47c-4e97-829d-e846f71f4213 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:54.312963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b0b7b-a47c-4e97-829d-e846f71f4213 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:54.339757 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:54.351406 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c536e-e0fa-4629-8f00-f2356cdfbfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:54.352034 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c536e-e0fa-4629-8f00-f2356cdfbfdc None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:54.352318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c536e-e0fa-4629-8f00-f2356cdfbfdc None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:54.352596 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c536e-e0fa-4629-8f00-f2356cdfbfdc None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:54.353389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c536e-e0fa-4629-8f00-f2356cdfbfdc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:54.354819 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c536e-e0fa-4629-8f00-f2356cdfbfdc None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:54.389149 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 512/1017] 10.222.0.100 () {60 vars in 1146 bytes} [Tue Aug 2 08:58:54 2022] POST /identity/v3/projects => generated 312 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:58:54.460680 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903e481-5c84-4229-8b2f-2f2445b3a406 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:54.460885 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903e481-5c84-4229-8b2f-2f2445b3a406 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:54.461224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903e481-5c84-4229-8b2f-2f2445b3a406 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:54.462676 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 506/1018] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:54.468521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb36489-807e-4ff0-a6b0-6c686a10856a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:54.468856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb36489-807e-4ff0-a6b0-6c686a10856a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:54.469092 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb36489-807e-4ff0-a6b0-6c686a10856a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:54.488689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1cb36489-807e-4ff0-a6b0-6c686a10856a None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:54.489982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1cb36489-807e-4ff0-a6b0-6c686a10856a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:54.566662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 513/1019] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:54 2022] POST /identity/v3/auth/tokens => generated 1877 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:54.589770 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903e481-5c84-4229-8b2f-2f2445b3a406 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:54.650631 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:54.701356 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903e481-5c84-4229-8b2f-2f2445b3a406 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:54.735986 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e437603d-1d9d-4cc5-9cb4-099bab296254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:54.736526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437603d-1d9d-4cc5-9cb4-099bab296254 None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:54.736810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437603d-1d9d-4cc5-9cb4-099bab296254 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:54.737086 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437603d-1d9d-4cc5-9cb4-099bab296254 None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:54.737808 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e437603d-1d9d-4cc5-9cb4-099bab296254 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:54.739173 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e437603d-1d9d-4cc5-9cb4-099bab296254 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:54.761136 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 507/1020] 10.222.0.100 () {60 vars in 1146 bytes} [Tue Aug 2 08:58:54 2022] POST /identity/v3/services => generated 243 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:58:55.838589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ff432-16fc-48a0-a873-c14cc9d50471 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:55.839126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ff432-16fc-48a0-a873-c14cc9d50471 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:55.839126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ff432-16fc-48a0-a873-c14cc9d50471 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:55.840736 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 514/1021] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:55.845795 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e86133-b03e-43aa-acd5-5041e3db7344 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:55.846182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e86133-b03e-43aa-acd5-5041e3db7344 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:55.846343 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e86133-b03e-43aa-acd5-5041e3db7344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:55.864085 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-f4e86133-b03e-43aa-acd5-5041e3db7344 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:55.865839 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f4e86133-b03e-43aa-acd5-5041e3db7344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:55.949971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 508/1022] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:55 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:55.972542 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ff432-16fc-48a0-a873-c14cc9d50471 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:56.075901 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ff432-16fc-48a0-a873-c14cc9d50471 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:56.083292 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:56.111758 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e16c8f-5aec-4c24-8838-3ac05a879f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:56.112328 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e16c8f-5aec-4c24-8838-3ac05a879f47 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:56.112609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e16c8f-5aec-4c24-8838-3ac05a879f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:56.112888 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e16c8f-5aec-4c24-8838-3ac05a879f47 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:56.127239 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e16c8f-5aec-4c24-8838-3ac05a879f47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:56.133184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-f9e16c8f-5aec-4c24-8838-3ac05a879f47 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:58:56.133184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:56.133184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:56.133184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:56.133184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:56.133184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:56.133184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:56.133184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:56.133184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:56.133184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:56.133184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:56.133184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:56.133184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:56.133184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:56.133184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:56.133184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:56.134181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:56.134181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:56.134181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:56.134181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:56.134181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:56.134181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:56.134181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:56.134181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:56.134181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:56.134181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:56.134181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:56.134181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:56.134181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:56.134181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:56.134181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:56.134181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:56.135085 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:56.135085 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:56.135085 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:56.135085 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:56.135085 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:56.135085 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:56.135085 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:56.135085 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:56.135085 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:56.135085 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:56.135085 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:58:56.135085 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:56.136579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 515/1023] 10.222.0.100 () {56 vars in 1125 bytes} [Tue Aug 2 08:58:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:56.140261 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e86133-b03e-43aa-acd5-5041e3db7344 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:56.183267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:56.246788 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e86133-b03e-43aa-acd5-5041e3db7344 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:56.280489 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba648e53-1a6d-4a26-9766-2153d442f1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:56.281123 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba648e53-1a6d-4a26-9766-2153d442f1ea None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:56.281417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba648e53-1a6d-4a26-9766-2153d442f1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:56.281693 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba648e53-1a6d-4a26-9766-2153d442f1ea None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:56.282400 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba648e53-1a6d-4a26-9766-2153d442f1ea None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:56.282706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba648e53-1a6d-4a26-9766-2153d442f1ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:56.284086 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba648e53-1a6d-4a26-9766-2153d442f1ea None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:56.297435 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 509/1024] 10.222.0.100 () {56 vars in 1120 bytes} [Tue Aug 2 08:58:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:56.298698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e16c8f-5aec-4c24-8838-3ac05a879f47 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:56.402193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e16c8f-5aec-4c24-8838-3ac05a879f47 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:56.436766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca9742-784f-48e5-8c8d-c5cf0f59cf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:56.437448 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca9742-784f-48e5-8c8d-c5cf0f59cf20 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:56.437729 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca9742-784f-48e5-8c8d-c5cf0f59cf20 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:56.438002 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca9742-784f-48e5-8c8d-c5cf0f59cf20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:56.448238 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca9742-784f-48e5-8c8d-c5cf0f59cf20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:56.449699 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca9742-784f-48e5-8c8d-c5cf0f59cf20 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:56.462524 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd213d-f3dd-4a98-b2b4-6ecaf244ab68 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:56.462814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd213d-f3dd-4a98-b2b4-6ecaf244ab68 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:56.463094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd213d-f3dd-4a98-b2b4-6ecaf244ab68 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:56.464614 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 510/1025] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:56.464730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-bbca9742-784f-48e5-8c8d-c5cf0f59cf20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:56.464730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:56.464730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:56.464730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:56.464730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:56.464730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:56.464730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:56.464730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:56.464730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:56.464730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:56.464730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:56.464730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:56.464730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:56.464730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:56.464730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:56.466030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:56.466030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:56.466030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:56.466030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:56.466030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:56.466030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:56.466030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:56.466030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:56.466030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:56.466030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:56.466030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:56.466030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:56.466030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:56.466030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:56.466030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:56.466030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:56.466030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:56.467262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:56.467262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:56.467262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:56.467262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:56.467919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 516/1026] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:56.471236 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca9742-784f-48e5-8c8d-c5cf0f59cf20 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:56.471502 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d9bb6-8aac-4542-99f7-f4cffd9304f8 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:56.471789 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d9bb6-8aac-4542-99f7-f4cffd9304f8 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:56.472116 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d9bb6-8aac-4542-99f7-f4cffd9304f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:56.491430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-db5d9bb6-8aac-4542-99f7-f4cffd9304f8 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:56.492614 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-db5d9bb6-8aac-4542-99f7-f4cffd9304f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:56.494938 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:56.565961 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 511/1027] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:56 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:58:56.590416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d9bb6-8aac-4542-99f7-f4cffd9304f8 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:56.591794 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca9742-784f-48e5-8c8d-c5cf0f59cf20 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:56.622032 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:56.634280 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25b87f-c26b-44d4-8c96-5002f8f304e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:56.635122 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25b87f-c26b-44d4-8c96-5002f8f304e5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:56.635514 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25b87f-c26b-44d4-8c96-5002f8f304e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:56.635798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25b87f-c26b-44d4-8c96-5002f8f304e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:56.636501 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25b87f-c26b-44d4-8c96-5002f8f304e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:56.636811 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25b87f-c26b-44d4-8c96-5002f8f304e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:56.638322 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25b87f-c26b-44d4-8c96-5002f8f304e5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:56.652137 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 517/1028] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:56.660771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25b87f-c26b-44d4-8c96-5002f8f304e5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:56.709126 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d9bb6-8aac-4542-99f7-f4cffd9304f8 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:56.745163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed3bc2-9eeb-4d7f-85f6-2f4a126abb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:56.745816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed3bc2-9eeb-4d7f-85f6-2f4a126abb4f None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:56.746099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed3bc2-9eeb-4d7f-85f6-2f4a126abb4f None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:56.746373 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed3bc2-9eeb-4d7f-85f6-2f4a126abb4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:56.747542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed3bc2-9eeb-4d7f-85f6-2f4a126abb4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:56.748671 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:56.748985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed3bc2-9eeb-4d7f-85f6-2f4a126abb4f None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:56.750907 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 512/1029] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:56 2022] GET /identity/v3/domains/default => generated 200 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:56.756686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed3bc2-9eeb-4d7f-85f6-2f4a126abb4f None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:56.778497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25b87f-c26b-44d4-8c96-5002f8f304e5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:56.814695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf686137-25f2-486a-a208-f06bdfb70960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:56.815338 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf686137-25f2-486a-a208-f06bdfb70960 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:56.815624 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf686137-25f2-486a-a208-f06bdfb70960 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:56.815900 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf686137-25f2-486a-a208-f06bdfb70960 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:56.823620 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf686137-25f2-486a-a208-f06bdfb70960 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:56.825048 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf686137-25f2-486a-a208-f06bdfb70960 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:56.836689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-bf686137-25f2-486a-a208-f06bdfb70960 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:58:56.836689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:56.836689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:56.836689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:56.836689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:56.836689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:56.836689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:56.836689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:56.836689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:56.836689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:56.836689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:56.836689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:56.836689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:58:56.836689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:58:56.836689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:58:56.836689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:58:56.837917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:56.837917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:56.837917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:58:56.837917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:56.837917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:58:56.837917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:58:56.837917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:56.837917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:56.837917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:56.837917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:56.837917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:56.837917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:56.837917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:56.837917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:56.837917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:56.837917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:56.837917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:56.837917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:56.839525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:56.839525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:56.839525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:58:56.839525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:58:56.839525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:58:56.839525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:58:56.839525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:58:56.839525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:58:56.839525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:58:56.839525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:56.839525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 518/1030] 10.222.0.100 () {56 vars in 1098 bytes} [Tue Aug 2 08:58:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:56.839525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b4669-4d5a-407c-989a-64cd65905cec None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:56.840326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b4669-4d5a-407c-989a-64cd65905cec None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:56.840326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b4669-4d5a-407c-989a-64cd65905cec None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:56.841308 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 519/1031] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:56.843312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b4669-4d5a-407c-989a-64cd65905cec None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:56.861841 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:56.866521 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed3bc2-9eeb-4d7f-85f6-2f4a126abb4f None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:56.900970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02853d-17d0-4c02-af2a-aa160d18c7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:56.901616 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02853d-17d0-4c02-af2a-aa160d18c7be None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:56.901907 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02853d-17d0-4c02-af2a-aa160d18c7be None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:56.902183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02853d-17d0-4c02-af2a-aa160d18c7be None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:56.902860 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c02853d-17d0-4c02-af2a-aa160d18c7be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:56.904300 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c02853d-17d0-4c02-af2a-aa160d18c7be None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:56.928615 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 513/1032] 10.222.0.100 () {60 vars in 1137 bytes} [Tue Aug 2 08:58:56 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 08:58:56.929977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf7641-fed7-4284-8ca7-5e8e26489a7d None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:56.930274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf7641-fed7-4284-8ca7-5e8e26489a7d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:56.930552 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf7641-fed7-4284-8ca7-5e8e26489a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:56.954703 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-e8cf7641-fed7-4284-8ca7-5e8e26489a7d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:56.956134 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e8cf7641-fed7-4284-8ca7-5e8e26489a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:56.967502 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b4669-4d5a-407c-989a-64cd65905cec None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:57.006579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a04319-1ce8-4218-aa5b-934b8abfe76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:57.007238 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a04319-1ce8-4218-aa5b-934b8abfe76a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:57.007521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a04319-1ce8-4218-aa5b-934b8abfe76a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:57.007800 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a04319-1ce8-4218-aa5b-934b8abfe76a None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:57.008609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a04319-1ce8-4218-aa5b-934b8abfe76a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:57.008926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a04319-1ce8-4218-aa5b-934b8abfe76a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:57.010429 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a04319-1ce8-4218-aa5b-934b8abfe76a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:57.019586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:57.031099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 520/1033] 10.222.0.100 () {56 vars in 1138 bytes} [Tue Aug 2 08:58:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:57.037575 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a04319-1ce8-4218-aa5b-934b8abfe76a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:57.044380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 514/1034] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:56 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:57.069599 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf7641-fed7-4284-8ca7-5e8e26489a7d None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:57.151093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a04319-1ce8-4218-aa5b-934b8abfe76a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:57.167516 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:57.186990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf7641-fed7-4284-8ca7-5e8e26489a7d None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:57.189759 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf52e9c-0013-422f-b08f-41d449efabe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:57.190574 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf52e9c-0013-422f-b08f-41d449efabe6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:57.190860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf52e9c-0013-422f-b08f-41d449efabe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:57.191142 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf52e9c-0013-422f-b08f-41d449efabe6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:57.226071 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3806af6-657d-46e2-849a-219097ea2b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:57.226711 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3806af6-657d-46e2-849a-219097ea2b0c None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:57.226990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3806af6-657d-46e2-849a-219097ea2b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:57.227261 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3806af6-657d-46e2-849a-219097ea2b0c None admin] PATH_INFO: `/v3/services/image` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:57.227882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3806af6-657d-46e2-849a-219097ea2b0c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:57.229282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3806af6-657d-46e2-849a-219097ea2b0c None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:57.238613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-e3806af6-657d-46e2-849a-219097ea2b0c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 08:58:57.238613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:57.238613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:58:57.238613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:58:57.238613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:58:57.238613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:58:57.238613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:58:57.238613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:57.238613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 08:58:57.238613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:57.238613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:58:57.238613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:57.238613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:57.238613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:57.238613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:57.238613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:57.238613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:57.240267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:57.240267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:57.240267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:57.240267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:57.240267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:57.240267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:57.240267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:58:57.240267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:58:57.240267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:58:57.240267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:58:57.240267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:57.240267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:57.240267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:57.240267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:57.240267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:57.240267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:57.241349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:57.241349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:57.241349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:57.241349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:57.241349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:57.241349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:57.241349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:57.241349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:57.241349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:57.241349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:57.241349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:58:57.241349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:58:57.241349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 08:58:57.241349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:57.241349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 515/1035] 10.222.0.100 () {56 vars in 1110 bytes} [Tue Aug 2 08:58:57 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:57.246320 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3806af6-657d-46e2-849a-219097ea2b0c None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:57.262278 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:57.270364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf52e9c-0013-422f-b08f-41d449efabe6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:57.272492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf52e9c-0013-422f-b08f-41d449efabe6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:57.372157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3806af6-657d-46e2-849a-219097ea2b0c None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:57.381513 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 521/1036] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:58:57 2022] GET /identity/v3/auth/tokens => generated 1973 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 08:58:57.383680 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf52e9c-0013-422f-b08f-41d449efabe6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:57.408716 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1b32c-cfb0-477d-99be-c165e576c73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:57.409362 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1b32c-cfb0-477d-99be-c165e576c73b None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:57.409641 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1b32c-cfb0-477d-99be-c165e576c73b None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:57.410119 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1b32c-cfb0-477d-99be-c165e576c73b None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:57.410584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1b32c-cfb0-477d-99be-c165e576c73b None admin] RBAC: Adding query filter params (name=image) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:57.410889 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1b32c-cfb0-477d-99be-c165e576c73b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:57.412425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1b32c-cfb0-477d-99be-c165e576c73b None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:57.417449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 516/1037] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:57 2022] GET /identity/v3/services?name=image => generated 123 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:57.423061 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1b32c-cfb0-477d-99be-c165e576c73b None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:57.439042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:57.491985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:57.498103 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf52e9c-0013-422f-b08f-41d449efabe6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:57.529885 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1b32c-cfb0-477d-99be-c165e576c73b None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:57.534665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6ffb4-105c-4cb8-ac2f-e8f8ca9797d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:57.535313 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6ffb4-105c-4cb8-ac2f-e8f8ca9797d2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:57.535590 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6ffb4-105c-4cb8-ac2f-e8f8ca9797d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:57.535863 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6ffb4-105c-4cb8-ac2f-e8f8ca9797d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:57.548577 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae6ffb4-105c-4cb8-ac2f-e8f8ca9797d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:57.552808 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae6ffb4-105c-4cb8-ac2f-e8f8ca9797d2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:57.568469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0079c9-e2c9-418b-8ed6-14753ed124ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:57.568902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-bae6ffb4-105c-4cb8-ac2f-e8f8ca9797d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:57.568902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:57.568902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:57.568902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:57.568902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:57.568902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:57.568902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:57.568902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:57.568902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:57.568902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:57.568902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:57.568902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:57.568902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:58:57.568902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:58:57.568902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:58:57.569912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:58:57.569912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:57.569912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:57.569912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:57.569912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:57.569912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:57.569912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:57.569912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:57.569912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:57.569912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:57.569912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:57.569912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:57.569912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:57.569912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:57.569912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:57.569912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:57.569912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:57.570825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:58:57.570825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:58:57.570825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:58:57.570825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:57.571058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0079c9-e2c9-418b-8ed6-14753ed124ef None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:57.571058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0079c9-e2c9-418b-8ed6-14753ed124ef None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:57.571058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0079c9-e2c9-418b-8ed6-14753ed124ef None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:57.571058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0079c9-e2c9-418b-8ed6-14753ed124ef None admin] RBAC: Adding query filter params (type=image) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:57.571058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0079c9-e2c9-418b-8ed6-14753ed124ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:57.572015 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0079c9-e2c9-418b-8ed6-14753ed124ef None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:57.575134 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 522/1038] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:58:57.575964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6ffb4-105c-4cb8-ac2f-e8f8ca9797d2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:57.582065 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 517/1039] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:58:57 2022] GET /identity/v3/services?type=image => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:57.584164 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0079c9-e2c9-418b-8ed6-14753ed124ef None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:57.666245 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:57.690850 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6ffb4-105c-4cb8-ac2f-e8f8ca9797d2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:57.694448 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0079c9-e2c9-418b-8ed6-14753ed124ef None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:57.728814 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124222fd-33ec-4283-ac57-0bc8090bf1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:57.729485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124222fd-33ec-4283-ac57-0bc8090bf1fd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:57.729773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124222fd-33ec-4283-ac57-0bc8090bf1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:57.730054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124222fd-33ec-4283-ac57-0bc8090bf1fd None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:57.730775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124222fd-33ec-4283-ac57-0bc8090bf1fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:57.731096 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124222fd-33ec-4283-ac57-0bc8090bf1fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:57.732452 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124222fd-33ec-4283-ac57-0bc8090bf1fd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:57.733496 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d5e4e-af2c-4f6e-9fc2-5c7f3789797e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:57.734321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d5e4e-af2c-4f6e-9fc2-5c7f3789797e None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:57.734774 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d5e4e-af2c-4f6e-9fc2-5c7f3789797e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:57.735209 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d5e4e-af2c-4f6e-9fc2-5c7f3789797e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:57.735961 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d5e4e-af2c-4f6e-9fc2-5c7f3789797e None admin] RBAC: Adding query filter params (interface=public, service_id=14d6103a8e82423a98da0de6395b4f5e, region_id=RegionOne) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:57.736271 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d5e4e-af2c-4f6e-9fc2-5c7f3789797e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:57.737651 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d5e4e-af2c-4f6e-9fc2-5c7f3789797e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:57.750391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 523/1040] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:58:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:57.750565 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 518/1041] 10.222.0.100 () {56 vars in 1256 bytes} [Tue Aug 2 08:58:57 2022] GET /identity/v3/endpoints?service_id=14d6103a8e82423a98da0de6395b4f5e&interface=public®ion_id=RegionOne => generated 195 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:57.750678 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124222fd-33ec-4283-ac57-0bc8090bf1fd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:57.751369 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d5e4e-af2c-4f6e-9fc2-5c7f3789797e None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:57.761262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:57.851951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124222fd-33ec-4283-ac57-0bc8090bf1fd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:57.859431 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d5e4e-af2c-4f6e-9fc2-5c7f3789797e None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:57.887372 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e38fb-c1a7-4162-b488-0b88d74f8a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:57.887985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e38fb-c1a7-4162-b488-0b88d74f8a3b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:57.888265 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e38fb-c1a7-4162-b488-0b88d74f8a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:57.888544 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e38fb-c1a7-4162-b488-0b88d74f8a3b None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:57.889219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e38fb-c1a7-4162-b488-0b88d74f8a3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:57.890567 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e38fb-c1a7-4162-b488-0b88d74f8a3b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:57.894565 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a8680-ea6b-4cc4-a971-aa2cba1131c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:57.895203 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a8680-ea6b-4cc4-a971-aa2cba1131c5 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:57.896031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a8680-ea6b-4cc4-a971-aa2cba1131c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:57.896320 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a8680-ea6b-4cc4-a971-aa2cba1131c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:57.905548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 524/1042] 10.222.0.100 () {56 vars in 1092 bytes} [Tue Aug 2 08:58:57 2022] GET /identity/v3/services => generated 1966 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 08:58:57.908229 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a8680-ea6b-4cc4-a971-aa2cba1131c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:57.909784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a8680-ea6b-4cc4-a971-aa2cba1131c5 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:57.913841 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:57.927910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-125a8680-ea6b-4cc4-a971-aa2cba1131c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:57.927910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:57.927910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:57.927910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:57.927910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:57.927910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:57.927910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:57.927910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:57.927910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:57.927910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:57.927910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:57.927910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:57.927910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:58:57.927910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:58:57.927910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:58:57.928892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:58:57.928892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:57.928892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:57.928892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:57.928892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:57.928892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:57.928892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:57.928892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:57.928892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:57.928892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:57.928892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:57.928892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:57.928892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:57.928892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:57.928892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:57.928892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:57.928892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:57.934430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:58:57.934430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:58:57.934430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:58:57.934430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:58:57.934430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:58:57.934430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:58:57.934430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:58:57.934430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:58:57.934430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 519/1043] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:58:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:57.935513 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e38fb-c1a7-4162-b488-0b88d74f8a3b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:58.014837 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:58.049427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e38fb-c1a7-4162-b488-0b88d74f8a3b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:58.087232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03d9df-98e4-48d7-b008-186a0b00486a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:58.087914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03d9df-98e4-48d7-b008-186a0b00486a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:58.088193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03d9df-98e4-48d7-b008-186a0b00486a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:58.088512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03d9df-98e4-48d7-b008-186a0b00486a None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:58.089284 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03d9df-98e4-48d7-b008-186a0b00486a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:58.089592 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03d9df-98e4-48d7-b008-186a0b00486a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:58.091000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03d9df-98e4-48d7-b008-186a0b00486a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:58.109206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 525/1044] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:58:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:58.110591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a8680-ea6b-4cc4-a971-aa2cba1131c5 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:58.212595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a8680-ea6b-4cc4-a971-aa2cba1131c5 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:58.246392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6d14b-76b9-4476-a0fb-b78ed2a1af97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:58.247002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6d14b-76b9-4476-a0fb-b78ed2a1af97 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:58.247282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6d14b-76b9-4476-a0fb-b78ed2a1af97 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:58.247551 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6d14b-76b9-4476-a0fb-b78ed2a1af97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:58.248325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6d14b-76b9-4476-a0fb-b78ed2a1af97 None admin] RBAC: Adding query filter params (role.id=c3082fcfa7e74f6a91ca6cdedd44f0ac, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=49a0c1a4a6cf43e28ec1b8488c681848) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:58.248632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6d14b-76b9-4476-a0fb-b78ed2a1af97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:58.250334 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6d14b-76b9-4476-a0fb-b78ed2a1af97 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:58.261208 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 520/1045] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:58:58 2022] GET /identity/v3/role_assignments?user.id=49a0c1a4a6cf43e28ec1b8488c681848&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=c3082fcfa7e74f6a91ca6cdedd44f0ac => generated 260 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:59.010052 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869a422-80db-43e4-8049-dad2436aa90d None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:59.010559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869a422-80db-43e4-8049-dad2436aa90d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:59.010637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869a422-80db-43e4-8049-dad2436aa90d None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:59.012004 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 526/1046] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:59.017703 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18860e37-15ae-4858-8472-5db0c8f2e1ff None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:59.018255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18860e37-15ae-4858-8472-5db0c8f2e1ff None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:59.018255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18860e37-15ae-4858-8472-5db0c8f2e1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:59.038375 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-18860e37-15ae-4858-8472-5db0c8f2e1ff None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:59.039546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-18860e37-15ae-4858-8472-5db0c8f2e1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:59.100976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:59.110476 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 521/1047] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:59 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:58:59.133661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869a422-80db-43e4-8049-dad2436aa90d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:59.235438 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869a422-80db-43e4-8049-dad2436aa90d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:59.265552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:59.272320 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34cff8-02e5-4d18-989f-d50481f1bd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:59.272967 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34cff8-02e5-4d18-989f-d50481f1bd53 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:59.273606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34cff8-02e5-4d18-989f-d50481f1bd53 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:59.273881 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34cff8-02e5-4d18-989f-d50481f1bd53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:59.284002 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34cff8-02e5-4d18-989f-d50481f1bd53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:58:59.289066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-2d34cff8-02e5-4d18-989f-d50481f1bd53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:58:59.289066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:58:59.289066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:58:59.289066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:58:59.289066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:58:59.289066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:58:59.289066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:58:59.289066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:58:59.289066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:58:59.289066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:58:59.289066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:58:59.289066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:58:59.289066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:58:59.289066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:58:59.289066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:58:59.289066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:58:59.290119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:58:59.290119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:58:59.290119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:58:59.290119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:58:59.290119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:58:59.290119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:58:59.290119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:58:59.290119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:58:59.290119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:58:59.290119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:58:59.290119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:58:59.290119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:58:59.290119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:58:59.290119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:58:59.290119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:58:59.290119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:58:59.290987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:58:59.290987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:58:59.290987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:58:59.290987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:58:59.290987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:58:59.290987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:58:59.290987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:58:59.290987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:58:59.290987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:58:59.290987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:58:59.290987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:58:59.290987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:58:59.293462 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 527/1048] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:58:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:58:59.296118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18860e37-15ae-4858-8472-5db0c8f2e1ff None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:59.401466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18860e37-15ae-4858-8472-5db0c8f2e1ff None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:59.434726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc321fb-4eb6-481c-9648-6450b97be1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:59.435345 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc321fb-4eb6-481c-9648-6450b97be1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:59.435630 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc321fb-4eb6-481c-9648-6450b97be1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:59.435907 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc321fb-4eb6-481c-9648-6450b97be1f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:59.436625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc321fb-4eb6-481c-9648-6450b97be1f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:58:59.436936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc321fb-4eb6-481c-9648-6450b97be1f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:59.438344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc321fb-4eb6-481c-9648-6450b97be1f6 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:59.451531 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 522/1049] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:58:59 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:58:59.452970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34cff8-02e5-4d18-989f-d50481f1bd53 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:59.556694 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34cff8-02e5-4d18-989f-d50481f1bd53 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:59.590120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade160d-0a4a-42d6-a97e-a52a1da0541c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:59.590766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade160d-0a4a-42d6-a97e-a52a1da0541c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:59.591046 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade160d-0a4a-42d6-a97e-a52a1da0541c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:59.591319 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade160d-0a4a-42d6-a97e-a52a1da0541c None admin] PATH_INFO: `/v3/users/b8113783e8a54ddeb5b56bc696986f9d` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:59.604550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade160d-0a4a-42d6-a97e-a52a1da0541c None admin] RBAC: Authorizing `identity:get_user(user_id=b8113783e8a54ddeb5b56bc696986f9d)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:59.606008 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade160d-0a4a-42d6-a97e-a52a1da0541c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:59.612379 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 528/1050] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:58:59 2022] GET /identity/v3/users/b8113783e8a54ddeb5b56bc696986f9d => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:58:59.613957 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc321fb-4eb6-481c-9648-6450b97be1f6 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:59.672729 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:59.719441 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc321fb-4eb6-481c-9648-6450b97be1f6 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:59.753007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855705c1-b935-4d2a-a8ff-93ce6c60bdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:59.753721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855705c1-b935-4d2a-a8ff-93ce6c60bdbd None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:59.754007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855705c1-b935-4d2a-a8ff-93ce6c60bdbd None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:59.754285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855705c1-b935-4d2a-a8ff-93ce6c60bdbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:59.822646 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855705c1-b935-4d2a-a8ff-93ce6c60bdbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:59.824027 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855705c1-b935-4d2a-a8ff-93ce6c60bdbd None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:59.878795 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:59.923480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009db532-5f83-41d0-a990-408078cfe52f None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:59.923771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009db532-5f83-41d0-a990-408078cfe52f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:59.924050 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009db532-5f83-41d0-a990-408078cfe52f None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:59.925574 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 529/1051] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:58:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:58:59.930788 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e71ac0-ad5b-48cf-bd25-11768529b0d4 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:59.935745 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e71ac0-ad5b-48cf-bd25-11768529b0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:59.935867 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 523/1052] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:58:59 2022] GET /identity/v3/auth/tokens => generated 1973 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:58:59.935966 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e71ac0-ad5b-48cf-bd25-11768529b0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:59.936748 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855705c1-b935-4d2a-a8ff-93ce6c60bdbd None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:59.951320 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a8e71ac0-ad5b-48cf-bd25-11768529b0d4 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:59.952770 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a8e71ac0-ad5b-48cf-bd25-11768529b0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:00.002602 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:00.026407 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 530/1053] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:58:59 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:59:00.042123 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855705c1-b935-4d2a-a8ff-93ce6c60bdbd None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:00.049072 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e71ac0-ad5b-48cf-bd25-11768529b0d4 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:00.078393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d0374-0e12-4c6a-939a-8622a621e3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:00.078950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d0374-0e12-4c6a-939a-8622a621e3ac None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:00.079234 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d0374-0e12-4c6a-939a-8622a621e3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:00.079516 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d0374-0e12-4c6a-939a-8622a621e3ac None admin] PATH_INFO: `/v3/projects/e187ee4c4dfa4496b94a7ffedfc55c84` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:00.080665 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d0374-0e12-4c6a-939a-8622a621e3ac None admin] RBAC: Authorizing `identity:get_project(project_id=e187ee4c4dfa4496b94a7ffedfc55c84)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:00.082141 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d0374-0e12-4c6a-939a-8622a621e3ac None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:00.084159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 524/1054] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:58:59 2022] GET /identity/v3/projects/e187ee4c4dfa4496b94a7ffedfc55c84 => generated 312 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:00.089925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d0374-0e12-4c6a-939a-8622a621e3ac None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:00.116248 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:00.178856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e71ac0-ad5b-48cf-bd25-11768529b0d4 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:00.192855 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d0374-0e12-4c6a-939a-8622a621e3ac None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:00.220843 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68bafc-8873-4038-9fdb-4fe42771d6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:00.221931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68bafc-8873-4038-9fdb-4fe42771d6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:00.222402 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68bafc-8873-4038-9fdb-4fe42771d6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:00.223136 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68bafc-8873-4038-9fdb-4fe42771d6b5 None admin] PATH_INFO: `/v3/services/image` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:00.224447 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68bafc-8873-4038-9fdb-4fe42771d6b5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:00.226374 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68bafc-8873-4038-9fdb-4fe42771d6b5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:00.227624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345c58d-4dd4-44b4-bfea-9495c98e5c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:00.228246 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345c58d-4dd4-44b4-bfea-9495c98e5c02 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:00.228533 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345c58d-4dd4-44b4-bfea-9495c98e5c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:00.228816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345c58d-4dd4-44b4-bfea-9495c98e5c02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:00.229624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345c58d-4dd4-44b4-bfea-9495c98e5c02 None admin] RBAC: Adding query filter params (role.id=7cbff51867a7475d837afd51f6c12f11, scope.project.id=e187ee4c4dfa4496b94a7ffedfc55c84, user.id=b8113783e8a54ddeb5b56bc696986f9d) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:00.229937 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345c58d-4dd4-44b4-bfea-9495c98e5c02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:00.231363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345c58d-4dd4-44b4-bfea-9495c98e5c02 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:00.236839 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 525/1055] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:59:00 2022] GET /identity/v3/role_assignments?user.id=b8113783e8a54ddeb5b56bc696986f9d&scope.project.id=e187ee4c4dfa4496b94a7ffedfc55c84&role.id=7cbff51867a7475d837afd51f6c12f11 => generated 260 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:00.238378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-db68bafc-8873-4038-9fdb-4fe42771d6b5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 08:59:00.238378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:00.238378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:59:00.238378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:59:00.238378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:59:00.238378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:59:00.238378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:59:00.238378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:59:00.238378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 08:59:00.238378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:59:00.238378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:59:00.238378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:59:00.238378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:00.238378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:00.238378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:00.238378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:00.238378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:00.239924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:00.239924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:00.239924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:00.239924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:00.239924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:00.239924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:00.239924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:59:00.239924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:59:00.239924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:59:00.239924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:59:00.239924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:00.239924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:00.239924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:00.239924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:00.239924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:00.239924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:00.240810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:00.240810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:00.240810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:00.240810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:00.240810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:00.240810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:00.240810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:00.240810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:00.240810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:00.240810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:00.240810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:59:00.240810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:59:00.240810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 08:59:00.240810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:59:00.245318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 531/1056] 10.222.0.100 () {56 vars in 1110 bytes} [Tue Aug 2 08:59:00 2022] GET /identity/v3/services/image => generated 86 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:59:00.247026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345c58d-4dd4-44b4-bfea-9495c98e5c02 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:00.268000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc7118-c370-4de5-a966-abf784b40929 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:00.268327 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc7118-c370-4de5-a966-abf784b40929 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:00.268569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc7118-c370-4de5-a966-abf784b40929 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:00.270169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 532/1057] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:00 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:59:00.275950 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc08296-26af-4a1f-a056-a2a30dd77d41 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:00.276244 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc08296-26af-4a1f-a056-a2a30dd77d41 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:00.276513 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc08296-26af-4a1f-a056-a2a30dd77d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:00.295878 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-7bc08296-26af-4a1f-a056-a2a30dd77d41 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:00.297144 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7bc08296-26af-4a1f-a056-a2a30dd77d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:00.345290 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:00.350544 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:00.368226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345c58d-4dd4-44b4-bfea-9495c98e5c02 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:00.381536 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 533/1058] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:00 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:59:00.401970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc08296-26af-4a1f-a056-a2a30dd77d41 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:00.415505 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1548b6-186d-4c15-bcc7-88d5c4161367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:00.416128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1548b6-186d-4c15-bcc7-88d5c4161367 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:00.416407 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1548b6-186d-4c15-bcc7-88d5c4161367 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:00.416679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1548b6-186d-4c15-bcc7-88d5c4161367 None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:00.417427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1548b6-186d-4c15-bcc7-88d5c4161367 None admin] RBAC: Adding query filter params (name=image) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:00.417731 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1548b6-186d-4c15-bcc7-88d5c4161367 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:00.419148 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1548b6-186d-4c15-bcc7-88d5c4161367 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:00.424239 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 526/1059] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:59:00 2022] GET /identity/v3/services?name=image => generated 123 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:00.430035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1548b6-186d-4c15-bcc7-88d5c4161367 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:00.512523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc08296-26af-4a1f-a056-a2a30dd77d41 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:00.540252 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1548b6-186d-4c15-bcc7-88d5c4161367 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:00.548370 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12cb8c-4d9e-45c3-a5cc-370d9ccf9e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:00.549040 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12cb8c-4d9e-45c3-a5cc-370d9ccf9e2a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:00.549370 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12cb8c-4d9e-45c3-a5cc-370d9ccf9e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:00.549697 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12cb8c-4d9e-45c3-a5cc-370d9ccf9e2a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:00.560350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12cb8c-4d9e-45c3-a5cc-370d9ccf9e2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:59:00.565263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-9e12cb8c-4d9e-45c3-a5cc-370d9ccf9e2a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:59:00.565263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:00.565263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:00.565263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:00.565263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:00.565263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:00.565263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:00.565263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:00.565263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:00.565263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:00.565263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:00.565263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:00.565263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:59:00.565263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:59:00.565263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:59:00.565263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:59:00.566222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:59:00.566222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:59:00.566222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:59:00.566222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:59:00.566222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:00.566222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:00.566222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:00.566222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:00.566222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:00.566222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:00.566222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:00.566222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:00.566222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:00.566222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:00.566222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:00.566222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:00.572461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:00.572461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:00.572461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:00.572461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:00.572461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:59:00.572461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:59:00.572461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:59:00.572461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:59:00.572461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:59:00.572461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:59:00.572461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:59:00.572461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:59:00.572461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 534/1060] 10.222.0.100 () {56 vars in 1125 bytes} [Tue Aug 2 08:59:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:59:00.575383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12cb8c-4d9e-45c3-a5cc-370d9ccf9e2a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:00.582469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444ccda-a03d-4c77-8c74-f4f97ccfc4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:00.583667 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444ccda-a03d-4c77-8c74-f4f97ccfc4ed None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:00.583947 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444ccda-a03d-4c77-8c74-f4f97ccfc4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:00.584221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444ccda-a03d-4c77-8c74-f4f97ccfc4ed None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:00.584905 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444ccda-a03d-4c77-8c74-f4f97ccfc4ed None admin] RBAC: Adding query filter params (type=image) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:00.585238 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444ccda-a03d-4c77-8c74-f4f97ccfc4ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:00.586612 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444ccda-a03d-4c77-8c74-f4f97ccfc4ed None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:00.593453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 527/1061] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:59:00 2022] GET /identity/v3/services?type=image => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:59:00.599499 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444ccda-a03d-4c77-8c74-f4f97ccfc4ed None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:00.619781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:00.621884 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:00.704178 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12cb8c-4d9e-45c3-a5cc-370d9ccf9e2a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:00.706656 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444ccda-a03d-4c77-8c74-f4f97ccfc4ed None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:00.743571 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a51f3-f456-440d-835a-f414bc1b3484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:00.744191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a51f3-f456-440d-835a-f414bc1b3484 None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:00.744476 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a51f3-f456-440d-835a-f414bc1b3484 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:00.744577 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62f166-cf57-4256-8c18-51e43affafb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:00.744753 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a51f3-f456-440d-835a-f414bc1b3484 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:00.745235 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62f166-cf57-4256-8c18-51e43affafb0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:00.745506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a51f3-f456-440d-835a-f414bc1b3484 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:00.745613 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62f166-cf57-4256-8c18-51e43affafb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:00.745776 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62f166-cf57-4256-8c18-51e43affafb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:00.746478 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f62f166-cf57-4256-8c18-51e43affafb0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:00.746782 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f62f166-cf57-4256-8c18-51e43affafb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:00.746941 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a51f3-f456-440d-835a-f414bc1b3484 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:00.748124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f62f166-cf57-4256-8c18-51e43affafb0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:00.762065 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 535/1062] 10.222.0.100 () {56 vars in 1120 bytes} [Tue Aug 2 08:59:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:00.762819 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62f166-cf57-4256-8c18-51e43affafb0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:00.771117 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 528/1063] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 08:59:00 2022] POST /identity/v3/endpoints => generated 329 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 08:59:00.869850 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:00.877612 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62f166-cf57-4256-8c18-51e43affafb0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:00.928423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca8b3c-c0e1-41b8-88bd-1a9d50d2491f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:00.929081 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca8b3c-c0e1-41b8-88bd-1a9d50d2491f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:00.929389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca8b3c-c0e1-41b8-88bd-1a9d50d2491f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:00.929665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca8b3c-c0e1-41b8-88bd-1a9d50d2491f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:00.942662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca8b3c-c0e1-41b8-88bd-1a9d50d2491f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:00.944108 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca8b3c-c0e1-41b8-88bd-1a9d50d2491f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:00.960196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-e8ca8b3c-c0e1-41b8-88bd-1a9d50d2491f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:59:00.960196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:00.960196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:00.960196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:00.960196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:00.960196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:00.960196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:00.960196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:00.960196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:00.960196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:00.960196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:00.960196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:00.960196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:59:00.960196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:59:00.960196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:59:00.961193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:59:00.961193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:00.961193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:00.961193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:00.961193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:00.961193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:00.961193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:00.961193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:00.961193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:00.961193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:00.961193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:00.961193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:00.961193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:00.961193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:00.961193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:00.961193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:00.961193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:00.962118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:59:00.962118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:59:00.962118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:59:00.962118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:59:00.962118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 536/1064] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:59:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:59:00.968189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a51f3-f456-440d-835a-f414bc1b3484 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:01.044754 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:01.077862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a51f3-f456-440d-835a-f414bc1b3484 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:01.111558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b9a5d-27c0-4a18-830d-74801b0f7f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:01.112181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b9a5d-27c0-4a18-830d-74801b0f7f0d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:01.112466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b9a5d-27c0-4a18-830d-74801b0f7f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:01.112749 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b9a5d-27c0-4a18-830d-74801b0f7f0d None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:01.113465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b9a5d-27c0-4a18-830d-74801b0f7f0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:01.113775 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b9a5d-27c0-4a18-830d-74801b0f7f0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:01.115869 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b9a5d-27c0-4a18-830d-74801b0f7f0d None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:01.133063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 529/1065] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:59:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:59:01.134349 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca8b3c-c0e1-41b8-88bd-1a9d50d2491f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:01.234555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca8b3c-c0e1-41b8-88bd-1a9d50d2491f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:01.270551 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddd350-1178-45e3-974b-5eaa7bf9bf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:01.271164 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddd350-1178-45e3-974b-5eaa7bf9bf32 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:01.271498 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddd350-1178-45e3-974b-5eaa7bf9bf32 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:01.271780 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddd350-1178-45e3-974b-5eaa7bf9bf32 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:01.279327 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ddd350-1178-45e3-974b-5eaa7bf9bf32 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:01.280749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ddd350-1178-45e3-974b-5eaa7bf9bf32 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:01.291937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-84ddd350-1178-45e3-974b-5eaa7bf9bf32 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:59:01.291937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:01.291937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:01.291937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:01.291937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:01.291937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:01.291937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:01.291937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:01.291937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:01.291937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:01.291937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:01.291937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:01.291937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:59:01.291937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:59:01.291937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:59:01.291937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:59:01.292915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:01.292915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:01.292915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:59:01.292915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:59:01.292915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:59:01.292915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:59:01.292915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:01.292915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:01.292915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:01.292915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:01.292915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:01.292915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:01.292915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:01.292915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:01.292915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:01.292915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:01.292915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:01.292915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:01.298744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:01.298744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:01.298744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:59:01.298744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:59:01.298744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:59:01.298744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:59:01.298744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:59:01.298744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:59:01.298744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:59:01.298744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:59:01.298744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 537/1066] 10.222.0.100 () {56 vars in 1098 bytes} [Tue Aug 2 08:59:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:59:01.299351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b9a5d-27c0-4a18-830d-74801b0f7f0d None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:01.403199 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b9a5d-27c0-4a18-830d-74801b0f7f0d None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:01.431967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:01.438258 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0938c-0c9b-446c-993b-c1d134d224b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:01.438881 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0938c-0c9b-446c-993b-c1d134d224b9 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:01.439162 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0938c-0c9b-446c-993b-c1d134d224b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:01.439435 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0938c-0c9b-446c-993b-c1d134d224b9 None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:01.440226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0938c-0c9b-446c-993b-c1d134d224b9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:01.440535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0938c-0c9b-446c-993b-c1d134d224b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:01.441998 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0938c-0c9b-446c-993b-c1d134d224b9 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:01.465086 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 530/1067] 10.222.0.100 () {56 vars in 1138 bytes} [Tue Aug 2 08:59:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:59:01.466243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddd350-1178-45e3-974b-5eaa7bf9bf32 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:01.470167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:01.570975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddd350-1178-45e3-974b-5eaa7bf9bf32 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:01.604158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e098826-ce2b-46d0-ad47-6e3b40a04a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:01.604801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e098826-ce2b-46d0-ad47-6e3b40a04a5d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:01.605082 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e098826-ce2b-46d0-ad47-6e3b40a04a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:01.605384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e098826-ce2b-46d0-ad47-6e3b40a04a5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:01.615783 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e098826-ce2b-46d0-ad47-6e3b40a04a5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:01.617232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e098826-ce2b-46d0-ad47-6e3b40a04a5d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:01.632383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-6e098826-ce2b-46d0-ad47-6e3b40a04a5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:59:01.632383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:01.632383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:01.632383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:01.632383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:01.632383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:01.632383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:01.632383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:01.632383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:01.632383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:01.632383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:01.632383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:01.632383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:59:01.632383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:59:01.632383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:59:01.633545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:59:01.633545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:01.633545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:01.633545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:01.633545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:01.633545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:01.633545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:01.633545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:01.633545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:01.633545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:01.633545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:01.633545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:01.633545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:01.633545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:01.633545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:01.633545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:01.633545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:01.638999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:59:01.638999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:59:01.638999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:59:01.638999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:59:01.638999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 538/1068] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:59:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:59:01.639305 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0938c-0c9b-446c-993b-c1d134d224b9 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:01.740235 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0938c-0c9b-446c-993b-c1d134d224b9 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:01.774747 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8dd20-ef08-4d2b-9e6f-3c3a0c60ddef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:01.775371 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8dd20-ef08-4d2b-9e6f-3c3a0c60ddef None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:01.775652 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8dd20-ef08-4d2b-9e6f-3c3a0c60ddef None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:01.775926 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8dd20-ef08-4d2b-9e6f-3c3a0c60ddef None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:01.776611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a8dd20-ef08-4d2b-9e6f-3c3a0c60ddef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:01.776917 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a8dd20-ef08-4d2b-9e6f-3c3a0c60ddef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:01.778418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a8dd20-ef08-4d2b-9e6f-3c3a0c60ddef None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:01.793144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 531/1069] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:59:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:59:01.795787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e098826-ce2b-46d0-ad47-6e3b40a04a5d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:01.855163 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:01.906931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e098826-ce2b-46d0-ad47-6e3b40a04a5d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:01.946663 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ab6c4-cee3-4aff-8e37-93d4fc57403b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:01.947321 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ab6c4-cee3-4aff-8e37-93d4fc57403b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:01.947604 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ab6c4-cee3-4aff-8e37-93d4fc57403b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:01.947881 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ab6c4-cee3-4aff-8e37-93d4fc57403b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:01.955093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ab6c4-cee3-4aff-8e37-93d4fc57403b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:01.956517 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ab6c4-cee3-4aff-8e37-93d4fc57403b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:01.968654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-0f1ab6c4-cee3-4aff-8e37-93d4fc57403b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:59:01.968654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:01.968654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:01.968654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:01.968654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:01.968654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:01.968654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:01.968654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:01.968654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:01.968654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:01.968654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:01.968654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:01.968654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:59:01.968654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:59:01.968654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:59:01.969835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:59:01.969835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:01.969835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:01.969835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:01.969835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:01.969835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:01.969835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:01.969835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:01.969835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:01.969835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:01.969835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:01.969835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:01.969835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:01.969835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:01.969835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:01.969835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:01.969835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:01.971172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:59:01.971172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:59:01.971172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:59:01.971172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:59:01.971172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:59:01.971172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:59:01.971172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:59:01.971172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:59:01.971172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 539/1070] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:59:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:59:01.978430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8dd20-ef08-4d2b-9e6f-3c3a0c60ddef None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:02.030433 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:02.081345 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8dd20-ef08-4d2b-9e6f-3c3a0c60ddef None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:02.114646 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3394e-973a-4d46-8fbe-1e78aa176153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:02.115263 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3394e-973a-4d46-8fbe-1e78aa176153 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:02.115542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3394e-973a-4d46-8fbe-1e78aa176153 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:02.115811 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3394e-973a-4d46-8fbe-1e78aa176153 None admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:02.116545 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3394e-973a-4d46-8fbe-1e78aa176153 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:02.116850 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3394e-973a-4d46-8fbe-1e78aa176153 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:02.118294 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3394e-973a-4d46-8fbe-1e78aa176153 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:02.131149 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 532/1071] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:59:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:02.136813 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ab6c4-cee3-4aff-8e37-93d4fc57403b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:02.237589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ab6c4-cee3-4aff-8e37-93d4fc57403b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:02.253956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:02.264548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e6e99-3f60-4a3e-9d9b-7dc82f7447dd None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:02.264832 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e6e99-3f60-4a3e-9d9b-7dc82f7447dd None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:02.265101 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e6e99-3f60-4a3e-9d9b-7dc82f7447dd None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:02.266501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 533/1072] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:59:02.271882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd14d4b-e498-4e00-8db4-8212355e26c5 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:02.272160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd14d4b-e498-4e00-8db4-8212355e26c5 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:02.272428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd14d4b-e498-4e00-8db4-8212355e26c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:02.275075 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3842de-f116-4151-9877-b99689b4261d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:02.275757 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3842de-f116-4151-9877-b99689b4261d None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:02.276047 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3842de-f116-4151-9877-b99689b4261d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:02.276332 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3842de-f116-4151-9877-b99689b4261d None admin] PATH_INFO: `/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/49a0c1a4a6cf43e28ec1b8488c681848/roles/c3082fcfa7e74f6a91ca6cdedd44f0ac` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:02.278724 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3842de-f116-4151-9877-b99689b4261d None admin] RBAC: Authorizing `identity:create_grant(project_id=61ca0176324e445b94e42a5214d2c89b, user_id=49a0c1a4a6cf43e28ec1b8488c681848, role_id=c3082fcfa7e74f6a91ca6cdedd44f0ac)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:02.280150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3842de-f116-4151-9877-b99689b4261d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:02.293217 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-2fd14d4b-e498-4e00-8db4-8212355e26c5 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:02.293217 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-2fd14d4b-e498-4e00-8db4-8212355e26c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:02.297152 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 540/1073] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:59:02 2022] PUT /identity/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/49a0c1a4a6cf43e28ec1b8488c681848/roles/c3082fcfa7e74f6a91ca6cdedd44f0ac => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:59:02.375283 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 534/1074] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:02 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:59:02.399249 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3842de-f116-4151-9877-b99689b4261d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:02.512694 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3842de-f116-4151-9877-b99689b4261d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:02.547903 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce42c20-442f-4243-8107-d93bcdcffc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:02.548479 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce42c20-442f-4243-8107-d93bcdcffc38 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:02.548869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce42c20-442f-4243-8107-d93bcdcffc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:02.549101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce42c20-442f-4243-8107-d93bcdcffc38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:02.560288 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce42c20-442f-4243-8107-d93bcdcffc38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:59:02.566103 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-4ce42c20-442f-4243-8107-d93bcdcffc38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:59:02.566103 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:02.566103 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:02.566103 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:02.566103 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:02.566103 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:02.566103 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:02.566103 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:02.566103 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:02.566103 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:02.566103 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:02.566103 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:02.566103 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:59:02.566103 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:59:02.566103 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:59:02.566103 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:59:02.567646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:59:02.567646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:59:02.567646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:59:02.567646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:59:02.567646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:02.567646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:02.567646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:02.567646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:02.567646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:02.567646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:02.567646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:02.567646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:02.567646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:02.567646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:02.567646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:02.567646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:02.568893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:02.568893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:02.568893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:02.568893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:02.568893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:59:02.568893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:59:02.568893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:59:02.568893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:59:02.568893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:59:02.568893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:59:02.568893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:59:02.568893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:59:02.568893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 541/1075] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:59:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:59:02.574024 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd14d4b-e498-4e00-8db4-8212355e26c5 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:02.618181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:02.678884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd14d4b-e498-4e00-8db4-8212355e26c5 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:02.721728 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542fe01-73e6-4dea-9ab6-b1ee4f9cadfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:02.722354 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542fe01-73e6-4dea-9ab6-b1ee4f9cadfd None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:02.722636 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542fe01-73e6-4dea-9ab6-b1ee4f9cadfd None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:02.722914 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542fe01-73e6-4dea-9ab6-b1ee4f9cadfd None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:02.724196 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542fe01-73e6-4dea-9ab6-b1ee4f9cadfd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:02.724507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542fe01-73e6-4dea-9ab6-b1ee4f9cadfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:02.725902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542fe01-73e6-4dea-9ab6-b1ee4f9cadfd None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:02.746786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 535/1076] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:59:02 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:02.748424 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce42c20-442f-4243-8107-d93bcdcffc38 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:02.787129 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:02.826855 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e5e37-6e77-43b2-9eca-dbacb93e123d None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:02.827052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e5e37-6e77-43b2-9eca-dbacb93e123d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:02.827335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e5e37-6e77-43b2-9eca-dbacb93e123d None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:02.828825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 536/1077] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:59:02.834349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8a97e-dbb8-4135-94f8-22a78628d298 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:02.834643 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8a97e-dbb8-4135-94f8-22a78628d298 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:02.834929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8a97e-dbb8-4135-94f8-22a78628d298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:02.853151 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-c0f8a97e-dbb8-4135-94f8-22a78628d298 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:02.854358 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c0f8a97e-dbb8-4135-94f8-22a78628d298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:02.856631 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce42c20-442f-4243-8107-d93bcdcffc38 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:02.897386 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b696b40-759f-4f28-afa5-f4aa4b231f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:02.898176 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b696b40-759f-4f28-afa5-f4aa4b231f8a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:02.898564 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b696b40-759f-4f28-afa5-f4aa4b231f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:02.898959 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b696b40-759f-4f28-afa5-f4aa4b231f8a None admin] PATH_INFO: `/v3/users/b8113783e8a54ddeb5b56bc696986f9d` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:02.900227 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b696b40-759f-4f28-afa5-f4aa4b231f8a None admin] RBAC: Authorizing `identity:get_user(user_id=b8113783e8a54ddeb5b56bc696986f9d)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:02.904637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b696b40-759f-4f28-afa5-f4aa4b231f8a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:02.909272 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 542/1078] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:59:02 2022] GET /identity/v3/users/b8113783e8a54ddeb5b56bc696986f9d => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:02.912956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b696b40-759f-4f28-afa5-f4aa4b231f8a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:02.941900 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 537/1079] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:02 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:59:02.965293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8a97e-dbb8-4135-94f8-22a78628d298 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:02.995289 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:03.033733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b696b40-759f-4f28-afa5-f4aa4b231f8a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:03.037988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:03.073201 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8a97e-dbb8-4135-94f8-22a78628d298 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:03.074692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe29c6-0cfb-484a-8e1f-cbf417d12452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:03.075335 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe29c6-0cfb-484a-8e1f-cbf417d12452 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:03.076229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe29c6-0cfb-484a-8e1f-cbf417d12452 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:03.076229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe29c6-0cfb-484a-8e1f-cbf417d12452 None admin] PATH_INFO: `/v3/projects/e187ee4c4dfa4496b94a7ffedfc55c84` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:03.077191 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe29c6-0cfb-484a-8e1f-cbf417d12452 None admin] RBAC: Authorizing `identity:get_project(project_id=e187ee4c4dfa4496b94a7ffedfc55c84)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:03.078557 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe29c6-0cfb-484a-8e1f-cbf417d12452 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:03.080372 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 543/1080] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:59:02 2022] GET /identity/v3/projects/e187ee4c4dfa4496b94a7ffedfc55c84 => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:03.087046 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe29c6-0cfb-484a-8e1f-cbf417d12452 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:03.111845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bbce3-9489-493f-a268-8d3e812d53e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:03.112461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bbce3-9489-493f-a268-8d3e812d53e7 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:03.112740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bbce3-9489-493f-a268-8d3e812d53e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:03.113011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bbce3-9489-493f-a268-8d3e812d53e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:03.188169 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9bbce3-9489-493f-a268-8d3e812d53e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:03.189638 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9bbce3-9489-493f-a268-8d3e812d53e7 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:03.191366 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe29c6-0cfb-484a-8e1f-cbf417d12452 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:03.220586 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:03.228012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5b7d7-9bba-47c1-a5a6-b43203db805c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:03.228806 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5b7d7-9bba-47c1-a5a6-b43203db805c None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:03.229252 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5b7d7-9bba-47c1-a5a6-b43203db805c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:03.229548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5b7d7-9bba-47c1-a5a6-b43203db805c None admin] PATH_INFO: `/v3/projects/e187ee4c4dfa4496b94a7ffedfc55c84/users/b8113783e8a54ddeb5b56bc696986f9d/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:03.231906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5b7d7-9bba-47c1-a5a6-b43203db805c None admin] RBAC: Authorizing `identity:create_grant(project_id=e187ee4c4dfa4496b94a7ffedfc55c84, user_id=b8113783e8a54ddeb5b56bc696986f9d, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:03.233724 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5b7d7-9bba-47c1-a5a6-b43203db805c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:03.247749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 544/1081] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:59:03 2022] PUT /identity/v3/projects/e187ee4c4dfa4496b94a7ffedfc55c84/users/b8113783e8a54ddeb5b56bc696986f9d/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 08:59:03.304349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 538/1082] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:59:02 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 08:59:03.305720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5b7d7-9bba-47c1-a5a6-b43203db805c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:03.341294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:03.426222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5b7d7-9bba-47c1-a5a6-b43203db805c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:03.464663 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a15863-41c8-42c1-8406-6c3ad01e10e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:03.465357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a15863-41c8-42c1-8406-6c3ad01e10e3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:03.465644 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a15863-41c8-42c1-8406-6c3ad01e10e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:03.465922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a15863-41c8-42c1-8406-6c3ad01e10e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:03.476709 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a15863-41c8-42c1-8406-6c3ad01e10e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:03.478181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a15863-41c8-42c1-8406-6c3ad01e10e3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:03.493804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-e0a15863-41c8-42c1-8406-6c3ad01e10e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:59:03.493804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:03.493804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:03.493804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:03.493804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:03.493804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:03.493804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:03.493804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:03.493804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:03.493804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:03.493804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:03.493804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:03.493804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:59:03.493804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:59:03.493804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:59:03.494827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:59:03.494827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:03.494827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:03.494827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:03.494827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:03.494827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:03.494827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:03.494827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:03.494827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:03.494827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:03.494827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:03.494827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:03.494827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:03.494827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:03.494827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:03.494827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:03.494827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:03.495750 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:59:03.495750 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:59:03.495750 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:59:03.495750 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:59:03.495750 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 545/1083] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:59:03.501060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bbce3-9489-493f-a268-8d3e812d53e7 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:03.603341 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bbce3-9489-493f-a268-8d3e812d53e7 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:03.632596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:03.638959 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59bd0e-cc28-4d55-ab03-53ce262c594b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:03.639681 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59bd0e-cc28-4d55-ab03-53ce262c594b None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:03.640058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59bd0e-cc28-4d55-ab03-53ce262c594b None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:03.640428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59bd0e-cc28-4d55-ab03-53ce262c594b None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:03.641240 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59bd0e-cc28-4d55-ab03-53ce262c594b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:03.641645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59bd0e-cc28-4d55-ab03-53ce262c594b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:03.643100 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59bd0e-cc28-4d55-ab03-53ce262c594b None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:03.658456 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 539/1084] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:59:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:04.337440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924273f4-907e-48d5-9e97-f8ff5ef710a5 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:04.337964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924273f4-907e-48d5-9e97-f8ff5ef710a5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:04.338074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924273f4-907e-48d5-9e97-f8ff5ef710a5 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:04.339490 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 546/1085] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:59:04.344544 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61271910-6a62-479c-8510-8cad5616f76a None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:04.344972 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61271910-6a62-479c-8510-8cad5616f76a None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:04.345093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61271910-6a62-479c-8510-8cad5616f76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:04.362764 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-61271910-6a62-479c-8510-8cad5616f76a None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:04.364482 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-61271910-6a62-479c-8510-8cad5616f76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:04.433325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 540/1086] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:04 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:59:04.455531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924273f4-907e-48d5-9e97-f8ff5ef710a5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:04.540804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:04.558575 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924273f4-907e-48d5-9e97-f8ff5ef710a5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:04.592385 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399fe24-15d2-412d-8747-3fd7be4cef2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:04.592964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399fe24-15d2-412d-8747-3fd7be4cef2e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:04.593278 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399fe24-15d2-412d-8747-3fd7be4cef2e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:04.593553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399fe24-15d2-412d-8747-3fd7be4cef2e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:04.603530 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399fe24-15d2-412d-8747-3fd7be4cef2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:59:04.608480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-6399fe24-15d2-412d-8747-3fd7be4cef2e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:59:04.608480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:04.608480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:04.608480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:04.608480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:04.608480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:04.608480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:04.608480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:04.608480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:04.608480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:04.608480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:04.608480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:04.608480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:59:04.608480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:59:04.608480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:59:04.608480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:59:04.609606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:59:04.609606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:59:04.609606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:59:04.609606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:59:04.609606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:04.609606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:04.609606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:04.609606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:04.609606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:04.609606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:04.609606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:04.609606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:04.609606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:04.609606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:04.609606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:04.609606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:04.610503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:04.610503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:04.610503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:04.610503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:04.610503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:59:04.610503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:59:04.610503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:59:04.610503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:59:04.610503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:59:04.610503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:59:04.610503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:59:04.610503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:59:04.610503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 547/1087] 10.222.0.100 () {56 vars in 1125 bytes} [Tue Aug 2 08:59:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:59:04.615702 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61271910-6a62-479c-8510-8cad5616f76a None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:04.716710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61271910-6a62-479c-8510-8cad5616f76a None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:04.750014 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:04.762680 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1c778-bf05-49af-9f0e-68abf6d972a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:04.763421 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1c778-bf05-49af-9f0e-68abf6d972a0 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:04.763798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1c778-bf05-49af-9f0e-68abf6d972a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:04.764165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1c778-bf05-49af-9f0e-68abf6d972a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:04.764972 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb1c778-bf05-49af-9f0e-68abf6d972a0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:04.765395 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb1c778-bf05-49af-9f0e-68abf6d972a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:04.766867 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb1c778-bf05-49af-9f0e-68abf6d972a0 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:04.778270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 541/1088] 10.222.0.100 () {56 vars in 1120 bytes} [Tue Aug 2 08:59:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:04.783540 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399fe24-15d2-412d-8747-3fd7be4cef2e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:04.891074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399fe24-15d2-412d-8747-3fd7be4cef2e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:04.930705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae70f0-4e81-4807-8995-c361cfc8549c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:04.931368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae70f0-4e81-4807-8995-c361cfc8549c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:04.931654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae70f0-4e81-4807-8995-c361cfc8549c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:04.931932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae70f0-4e81-4807-8995-c361cfc8549c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:04.942707 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae70f0-4e81-4807-8995-c361cfc8549c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:04.944345 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae70f0-4e81-4807-8995-c361cfc8549c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:04.947147 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:04.962325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-01ae70f0-4e81-4807-8995-c361cfc8549c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:59:04.962325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:04.962325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:04.962325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:04.962325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:04.962325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:04.962325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:04.962325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:04.962325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:04.962325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:04.962325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:04.962325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:04.962325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:59:04.962325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:59:04.962325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:59:04.963312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:59:04.963312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:04.963312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:04.963312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:04.963312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:04.963312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:04.963312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:04.963312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:04.963312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:04.963312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:04.963312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:04.963312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:04.963312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:04.963312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:04.963312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:04.963312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:04.963312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:04.964224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:59:04.964224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:59:04.964224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:59:04.964224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:59:04.964224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 548/1089] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:59:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:59:04.969221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1c778-bf05-49af-9f0e-68abf6d972a0 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:05.071668 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1c778-bf05-49af-9f0e-68abf6d972a0 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:05.104714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890f639-239e-44c5-8a05-e053120dffd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:05.105347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890f639-239e-44c5-8a05-e053120dffd9 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:05.105628 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890f639-239e-44c5-8a05-e053120dffd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:05.105899 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890f639-239e-44c5-8a05-e053120dffd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:05.106574 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9890f639-239e-44c5-8a05-e053120dffd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:05.106876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9890f639-239e-44c5-8a05-e053120dffd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:05.108343 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9890f639-239e-44c5-8a05-e053120dffd9 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:05.120425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 542/1090] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:59:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:05.126261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae70f0-4e81-4807-8995-c361cfc8549c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:05.224497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae70f0-4e81-4807-8995-c361cfc8549c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:05.257561 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1b0ad-a9f7-48bf-af27-80fbe8a1e740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:05.258195 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1b0ad-a9f7-48bf-af27-80fbe8a1e740 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:05.258474 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1b0ad-a9f7-48bf-af27-80fbe8a1e740 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:05.258747 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1b0ad-a9f7-48bf-af27-80fbe8a1e740 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:05.266108 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1b0ad-a9f7-48bf-af27-80fbe8a1e740 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:05.267520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1b0ad-a9f7-48bf-af27-80fbe8a1e740 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:05.279255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-a4e1b0ad-a9f7-48bf-af27-80fbe8a1e740 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:59:05.279255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:05.279255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:05.279255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:05.279255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:05.279255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:05.279255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:05.279255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:05.279255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:05.279255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:05.279255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:05.279255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:05.279255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:59:05.279255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:59:05.279255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:59:05.279255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:59:05.280359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:05.280359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:05.280359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:59:05.280359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:59:05.280359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:59:05.280359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:59:05.280359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:05.280359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:05.280359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:05.280359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:05.280359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:05.280359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:05.280359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:05.280359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:05.280359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:05.280359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:05.280359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:05.280359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:05.281587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:05.281587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:05.281587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:59:05.281587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:59:05.281587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:59:05.281587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:59:05.281587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:59:05.281587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:59:05.281587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:59:05.281587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:59:05.285201 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 549/1091] 10.222.0.100 () {56 vars in 1098 bytes} [Tue Aug 2 08:59:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:59:05.286182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890f639-239e-44c5-8a05-e053120dffd9 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:05.297863 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be68ab8-0d13-44d5-9cd7-0b0c7daa6f69 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:05.298158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be68ab8-0d13-44d5-9cd7-0b0c7daa6f69 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:05.298442 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be68ab8-0d13-44d5-9cd7-0b0c7daa6f69 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:05.299784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 550/1092] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 08:59:05.304771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd3893-e911-4535-b383-a8b1d6d2da56 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:05.305065 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd3893-e911-4535-b383-a8b1d6d2da56 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:05.305369 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd3893-e911-4535-b383-a8b1d6d2da56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:05.324165 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a2dd3893-e911-4535-b383-a8b1d6d2da56 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:05.325410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a2dd3893-e911-4535-b383-a8b1d6d2da56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:05.333798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:05.375064 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:05.388870 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890f639-239e-44c5-8a05-e053120dffd9 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:05.392952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 551/1093] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:05 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:59:05.416046 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd3893-e911-4535-b383-a8b1d6d2da56 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:05.423299 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821f4b6-6ead-4a75-b336-da28b8bc2d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:05.423916 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821f4b6-6ead-4a75-b336-da28b8bc2d08 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:05.424193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821f4b6-6ead-4a75-b336-da28b8bc2d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:05.424464 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821f4b6-6ead-4a75-b336-da28b8bc2d08 None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:05.425277 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821f4b6-6ead-4a75-b336-da28b8bc2d08 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:05.425590 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821f4b6-6ead-4a75-b336-da28b8bc2d08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:05.427235 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821f4b6-6ead-4a75-b336-da28b8bc2d08 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:05.444514 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 543/1094] 10.222.0.100 () {56 vars in 1138 bytes} [Tue Aug 2 08:59:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:05.450197 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821f4b6-6ead-4a75-b336-da28b8bc2d08 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:05.515315 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd3893-e911-4535-b383-a8b1d6d2da56 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:05.546045 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821f4b6-6ead-4a75-b336-da28b8bc2d08 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:05.547570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:05.548536 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f6e43-efbe-4e39-b5e2-f0df2b18b8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:05.549215 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f6e43-efbe-4e39-b5e2-f0df2b18b8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:05.549502 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f6e43-efbe-4e39-b5e2-f0df2b18b8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:05.549784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f6e43-efbe-4e39-b5e2-f0df2b18b8e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:05.561298 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f6e43-efbe-4e39-b5e2-f0df2b18b8e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:59:05.566262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-2d9f6e43-efbe-4e39-b5e2-f0df2b18b8e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:59:05.566262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:05.566262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:05.566262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:05.566262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:05.566262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:05.566262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:05.566262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:05.566262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:05.566262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:05.566262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:05.566262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:05.566262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:59:05.566262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:59:05.566262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:59:05.566262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:59:05.567937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:59:05.567937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:59:05.567937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:59:05.567937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:59:05.567937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:05.567937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:05.567937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:05.567937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:05.567937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:05.567937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:05.567937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:05.567937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:05.567937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:05.567937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:05.567937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:05.567937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:05.568774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:05.568774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:05.568774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:05.568774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:05.568774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:59:05.568774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:59:05.568774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:59:05.568774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:59:05.568774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:59:05.568774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:59:05.568774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:59:05.568774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:59:05.568774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 552/1095] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:59:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:59:05.573941 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f6e43-efbe-4e39-b5e2-f0df2b18b8e2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:05.583398 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9005a7-fb3e-47ee-a313-e933e40bb63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:05.584179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9005a7-fb3e-47ee-a313-e933e40bb63c None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:05.584458 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9005a7-fb3e-47ee-a313-e933e40bb63c None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:05.584734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9005a7-fb3e-47ee-a313-e933e40bb63c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:05.621606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:05.651342 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9005a7-fb3e-47ee-a313-e933e40bb63c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:05.652779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9005a7-fb3e-47ee-a313-e933e40bb63c None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:05.674437 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f6e43-efbe-4e39-b5e2-f0df2b18b8e2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:05.708624 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fafad7-3a21-4f62-89e1-88857c7cca4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:05.709277 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fafad7-3a21-4f62-89e1-88857c7cca4f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:05.709561 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fafad7-3a21-4f62-89e1-88857c7cca4f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:05.709836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fafad7-3a21-4f62-89e1-88857c7cca4f None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:05.710541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fafad7-3a21-4f62-89e1-88857c7cca4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:05.710843 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fafad7-3a21-4f62-89e1-88857c7cca4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:05.712188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fafad7-3a21-4f62-89e1-88857c7cca4f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:05.720115 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 553/1096] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:59:05 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:59:05.725838 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fafad7-3a21-4f62-89e1-88857c7cca4f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:05.750451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 544/1097] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:59:05 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 08:59:05.755138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9005a7-fb3e-47ee-a313-e933e40bb63c None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:05.757033 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:05.826032 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fafad7-3a21-4f62-89e1-88857c7cca4f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:05.835127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:05.853038 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9005a7-fb3e-47ee-a313-e933e40bb63c None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:05.860971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269e73e-86c9-42e9-9247-e2bea2cdb56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:05.861648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269e73e-86c9-42e9-9247-e2bea2cdb56d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:05.861938 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269e73e-86c9-42e9-9247-e2bea2cdb56d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:05.862214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269e73e-86c9-42e9-9247-e2bea2cdb56d None admin] PATH_INFO: `/v3/users/b8113783e8a54ddeb5b56bc696986f9d` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:05.863380 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269e73e-86c9-42e9-9247-e2bea2cdb56d None admin] RBAC: Authorizing `identity:get_user(user_id=b8113783e8a54ddeb5b56bc696986f9d)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:05.864848 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269e73e-86c9-42e9-9247-e2bea2cdb56d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:05.866479 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 554/1098] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:59:05 2022] GET /identity/v3/users/b8113783e8a54ddeb5b56bc696986f9d => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:59:05.872163 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269e73e-86c9-42e9-9247-e2bea2cdb56d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:05.892043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb090cf7-aaea-43de-9996-24c62e379328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:05.892675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb090cf7-aaea-43de-9996-24c62e379328 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:05.892956 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb090cf7-aaea-43de-9996-24c62e379328 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:05.893261 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb090cf7-aaea-43de-9996-24c62e379328 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:05.903907 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb090cf7-aaea-43de-9996-24c62e379328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:05.905380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb090cf7-aaea-43de-9996-24c62e379328 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:05.921360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-fb090cf7-aaea-43de-9996-24c62e379328 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:59:05.921360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:05.921360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:05.921360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:05.921360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:05.921360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:05.921360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:05.921360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:05.921360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:05.921360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:05.921360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:05.921360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:05.921360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:59:05.921360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:59:05.921360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:59:05.922280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:59:05.922280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:05.922280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:05.922280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:05.922280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:05.922280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:05.922280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:05.922280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:05.922280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:05.922280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:05.922280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:05.922280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:05.922280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:05.922280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:05.922280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:05.922280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:05.922280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:05.923257 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:59:05.923257 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:59:05.923257 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:59:05.923257 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:59:05.923257 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 545/1099] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:59:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:59:05.928120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb090cf7-aaea-43de-9996-24c62e379328 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:05.978178 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269e73e-86c9-42e9-9247-e2bea2cdb56d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:05.983766 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:06.012106 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6210dd9-cf65-4a59-80e0-dde51fa7425f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:06.012720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6210dd9-cf65-4a59-80e0-dde51fa7425f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:06.012999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6210dd9-cf65-4a59-80e0-dde51fa7425f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:06.013302 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6210dd9-cf65-4a59-80e0-dde51fa7425f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:06.029222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb090cf7-aaea-43de-9996-24c62e379328 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:06.057689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:06.062153 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4920a57-f7e7-4d83-8411-4a71dc7e3d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:06.062764 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4920a57-f7e7-4d83-8411-4a71dc7e3d05 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:06.063043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4920a57-f7e7-4d83-8411-4a71dc7e3d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:06.063315 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4920a57-f7e7-4d83-8411-4a71dc7e3d05 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:06.063992 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4920a57-f7e7-4d83-8411-4a71dc7e3d05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:06.064295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4920a57-f7e7-4d83-8411-4a71dc7e3d05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:06.065683 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4920a57-f7e7-4d83-8411-4a71dc7e3d05 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:06.083202 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 546/1100] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 08:59:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:59:06.083335 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6210dd9-cf65-4a59-80e0-dde51fa7425f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:06.083335 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6210dd9-cf65-4a59-80e0-dde51fa7425f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:06.084120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4920a57-f7e7-4d83-8411-4a71dc7e3d05 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:06.181502 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4920a57-f7e7-4d83-8411-4a71dc7e3d05 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:06.182156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 555/1101] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:59:05 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 08:59:06.187993 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6210dd9-cf65-4a59-80e0-dde51fa7425f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:06.196644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:06.219004 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e92863-3e8a-4ae5-914d-505012230f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:06.219644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e92863-3e8a-4ae5-914d-505012230f5b None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:06.219929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e92863-3e8a-4ae5-914d-505012230f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:06.220205 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e92863-3e8a-4ae5-914d-505012230f5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:06.226877 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e92863-3e8a-4ae5-914d-505012230f5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:06.228349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e92863-3e8a-4ae5-914d-505012230f5b None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:06.240253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-f5e92863-3e8a-4ae5-914d-505012230f5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:59:06.240253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:06.240253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:06.240253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:06.240253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:06.240253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:06.240253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:06.240253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:06.240253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:06.240253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:06.240253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:06.240253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:06.240253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:59:06.240253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:59:06.240253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:59:06.241255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:59:06.241255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:06.241255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:06.241255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:06.241255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:06.241255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:06.241255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:06.241255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:06.241255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:06.241255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:06.241255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:06.241255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:06.241255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:06.241255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:06.241255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:06.241255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:06.241255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:06.242204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:59:06.242204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:59:06.242204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:59:06.242204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:59:06.242204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:59:06.242204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:59:06.242204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:59:06.242204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:59:06.242204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 547/1102] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 08:59:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:59:06.247226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e92863-3e8a-4ae5-914d-505012230f5b None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:06.263634 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:06.286894 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6210dd9-cf65-4a59-80e0-dde51fa7425f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:06.321552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9739b5-03be-4abf-9c9d-6aa2ae91f778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:06.322188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9739b5-03be-4abf-9c9d-6aa2ae91f778 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:06.322473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9739b5-03be-4abf-9c9d-6aa2ae91f778 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:06.322752 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9739b5-03be-4abf-9c9d-6aa2ae91f778 None admin] PATH_INFO: `/v3/projects/e187ee4c4dfa4496b94a7ffedfc55c84` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:06.323954 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9739b5-03be-4abf-9c9d-6aa2ae91f778 None admin] RBAC: Authorizing `identity:get_project(project_id=e187ee4c4dfa4496b94a7ffedfc55c84)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:06.325409 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9739b5-03be-4abf-9c9d-6aa2ae91f778 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:06.327075 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 556/1103] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:59:06 2022] GET /identity/v3/projects/e187ee4c4dfa4496b94a7ffedfc55c84 => generated 312 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:59:06.333556 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9739b5-03be-4abf-9c9d-6aa2ae91f778 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:06.347490 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e92863-3e8a-4ae5-914d-505012230f5b None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:06.381940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a87fca-ec39-4e3a-bacf-d11797379d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:06.382561 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a87fca-ec39-4e3a-bacf-d11797379d5a None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:06.382841 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a87fca-ec39-4e3a-bacf-d11797379d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:06.383117 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a87fca-ec39-4e3a-bacf-d11797379d5a None admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:06.383865 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a87fca-ec39-4e3a-bacf-d11797379d5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:06.384171 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a87fca-ec39-4e3a-bacf-d11797379d5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:06.385644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a87fca-ec39-4e3a-bacf-d11797379d5a None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:06.398279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 548/1104] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 08:59:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:06.404363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a87fca-ec39-4e3a-bacf-d11797379d5a None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:06.437191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:06.437433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9739b5-03be-4abf-9c9d-6aa2ae91f778 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:06.471336 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fa900-0d5d-454c-829f-0f681662de4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:06.471962 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fa900-0d5d-454c-829f-0f681662de4c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:06.472247 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fa900-0d5d-454c-829f-0f681662de4c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:06.472524 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fa900-0d5d-454c-829f-0f681662de4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:06.473573 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3fa900-0d5d-454c-829f-0f681662de4c None admin] RBAC: Adding query filter params (role.id=7cbff51867a7475d837afd51f6c12f11, scope.project.id=e187ee4c4dfa4496b94a7ffedfc55c84, user.id=b8113783e8a54ddeb5b56bc696986f9d) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:06.473895 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3fa900-0d5d-454c-829f-0f681662de4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:06.475317 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3fa900-0d5d-454c-829f-0f681662de4c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:06.489218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 557/1105] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:59:06 2022] GET /identity/v3/role_assignments?user.id=b8113783e8a54ddeb5b56bc696986f9d&scope.project.id=e187ee4c4dfa4496b94a7ffedfc55c84&role.id=7cbff51867a7475d837afd51f6c12f11 => generated 610 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:59:06.506363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a87fca-ec39-4e3a-bacf-d11797379d5a None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:06.543384 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8f7c2-a914-4278-8cf5-38b3e66d9d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:06.544013 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8f7c2-a914-4278-8cf5-38b3e66d9d3e None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:06.544297 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8f7c2-a914-4278-8cf5-38b3e66d9d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:06.544576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8f7c2-a914-4278-8cf5-38b3e66d9d3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:06.545413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8f7c2-a914-4278-8cf5-38b3e66d9d3e None admin] RBAC: Adding query filter params (role.id=c3082fcfa7e74f6a91ca6cdedd44f0ac, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=49a0c1a4a6cf43e28ec1b8488c681848) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:06.545730 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8f7c2-a914-4278-8cf5-38b3e66d9d3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:06.547204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8f7c2-a914-4278-8cf5-38b3e66d9d3e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:06.555866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 549/1106] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:59:06 2022] GET /identity/v3/role_assignments?user.id=49a0c1a4a6cf43e28ec1b8488c681848&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=c3082fcfa7e74f6a91ca6cdedd44f0ac => generated 610 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:59:08.493735 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a10044-017e-4a66-b185-5fef3792a6dd None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:08.494324 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a10044-017e-4a66-b185-5fef3792a6dd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:08.494324 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a10044-017e-4a66-b185-5fef3792a6dd None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:08.495623 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 558/1107] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:59:08.500881 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa30bf-0db2-47ff-98bf-88e416fc3696 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:08.501351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa30bf-0db2-47ff-98bf-88e416fc3696 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:08.501461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa30bf-0db2-47ff-98bf-88e416fc3696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:08.519650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-09fa30bf-0db2-47ff-98bf-88e416fc3696 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:08.520926 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-09fa30bf-0db2-47ff-98bf-88e416fc3696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:08.588505 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 550/1108] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:08 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:59:08.613669 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a10044-017e-4a66-b185-5fef3792a6dd None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:08.624976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:08.716529 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a10044-017e-4a66-b185-5fef3792a6dd None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:08.749712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6e384-7a46-4a5b-bc8e-149c4b991c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:08.750294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6e384-7a46-4a5b-bc8e-149c4b991c33 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:08.750579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6e384-7a46-4a5b-bc8e-149c4b991c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:08.750856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6e384-7a46-4a5b-bc8e-149c4b991c33 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:08.751998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6e384-7a46-4a5b-bc8e-149c4b991c33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:08.753447 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6e384-7a46-4a5b-bc8e-149c4b991c33 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:08.755074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 559/1109] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:59:08 2022] GET /identity/v3/domains/default => generated 200 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:08.760882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa30bf-0db2-47ff-98bf-88e416fc3696 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:08.788592 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:08.857120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa30bf-0db2-47ff-98bf-88e416fc3696 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:08.889492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36b55b-6529-4c45-ab14-9a1319e92412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:08.890088 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36b55b-6529-4c45-ab14-9a1319e92412 None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:08.890364 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36b55b-6529-4c45-ab14-9a1319e92412 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:08.890634 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36b55b-6529-4c45-ab14-9a1319e92412 None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:08.891297 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36b55b-6529-4c45-ab14-9a1319e92412 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:08.892709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36b55b-6529-4c45-ab14-9a1319e92412 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:08.913690 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 551/1110] 10.222.0.100 () {60 vars in 1137 bytes} [Tue Aug 2 08:59:08 2022] POST /identity/v3/users => generated 287 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 08:59:10.919849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfe972-985b-4a64-99c4-f86c9c45ed97 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:10.920218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfe972-985b-4a64-99c4-f86c9c45ed97 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:10.920314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfe972-985b-4a64-99c4-f86c9c45ed97 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:10.921665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 560/1111] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:59:10.926855 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3b960-5c96-4f51-b831-48d0308e2bc3 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:10.927313 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3b960-5c96-4f51-b831-48d0308e2bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:10.927417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3b960-5c96-4f51-b831-48d0308e2bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:10.945340 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-74c3b960-5c96-4f51-b831-48d0308e2bc3 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:10.946520 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-74c3b960-5c96-4f51-b831-48d0308e2bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:11.010965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 552/1112] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:10 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:59:11.034138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfe972-985b-4a64-99c4-f86c9c45ed97 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:11.109534 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:11.133578 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfe972-985b-4a64-99c4-f86c9c45ed97 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:11.165855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9e972-e35b-4f6c-b731-c458da6332c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:11.166480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9e972-e35b-4f6c-b731-c458da6332c5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:11.166756 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9e972-e35b-4f6c-b731-c458da6332c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:11.167027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9e972-e35b-4f6c-b731-c458da6332c5 None admin] PATH_INFO: `/v3/roles/bb986e49c38c40bab629ff32371b9358` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:11.168426 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9e972-e35b-4f6c-b731-c458da6332c5 None admin] RBAC: Authorizing `identity:get_role(role_id=bb986e49c38c40bab629ff32371b9358)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:11.169854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9e972-e35b-4f6c-b731-c458da6332c5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:11.171128 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 561/1113] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:59:11 2022] GET /identity/v3/roles/bb986e49c38c40bab629ff32371b9358 => generated 224 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:11.177654 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3b960-5c96-4f51-b831-48d0308e2bc3 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:11.192345 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:11.274507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3b960-5c96-4f51-b831-48d0308e2bc3 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:11.307675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038696d3-dbcd-4fbd-8e32-e53331f7e068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:11.308303 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038696d3-dbcd-4fbd-8e32-e53331f7e068 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:11.308584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038696d3-dbcd-4fbd-8e32-e53331f7e068 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:11.308862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038696d3-dbcd-4fbd-8e32-e53331f7e068 None admin] PATH_INFO: `/v3/users/2d80ea128f63433597c76804fc999bca` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:11.321486 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038696d3-dbcd-4fbd-8e32-e53331f7e068 None admin] RBAC: Authorizing `identity:get_user(user_id=2d80ea128f63433597c76804fc999bca)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:11.322918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038696d3-dbcd-4fbd-8e32-e53331f7e068 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:11.324525 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 553/1114] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:59:11 2022] GET /identity/v3/users/2d80ea128f63433597c76804fc999bca => generated 287 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:59:11.330218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9e972-e35b-4f6c-b731-c458da6332c5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:11.427648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9e972-e35b-4f6c-b731-c458da6332c5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:11.460223 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175d50f-ffb2-4292-870d-54f5c3a57ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:11.460828 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175d50f-ffb2-4292-870d-54f5c3a57ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:11.461134 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175d50f-ffb2-4292-870d-54f5c3a57ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:11.461407 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175d50f-ffb2-4292-870d-54f5c3a57ff9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:11.475695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:11.527845 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175d50f-ffb2-4292-870d-54f5c3a57ff9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:11.529295 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175d50f-ffb2-4292-870d-54f5c3a57ff9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:11.629810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 562/1115] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:59:11 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:59:11.635627 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038696d3-dbcd-4fbd-8e32-e53331f7e068 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:11.704254 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:11.733936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038696d3-dbcd-4fbd-8e32-e53331f7e068 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:11.767079 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2b5fe-0da5-4d00-8962-4ea80c2485c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:11.767732 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2b5fe-0da5-4d00-8962-4ea80c2485c7 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:11.768014 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2b5fe-0da5-4d00-8962-4ea80c2485c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:11.768289 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2b5fe-0da5-4d00-8962-4ea80c2485c7 None admin] PATH_INFO: `/v3/projects/e187ee4c4dfa4496b94a7ffedfc55c84` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:11.769466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2b5fe-0da5-4d00-8962-4ea80c2485c7 None admin] RBAC: Authorizing `identity:get_project(project_id=e187ee4c4dfa4496b94a7ffedfc55c84)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:11.771793 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2b5fe-0da5-4d00-8962-4ea80c2485c7 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:11.773487 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 554/1116] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:59:11 2022] GET /identity/v3/projects/e187ee4c4dfa4496b94a7ffedfc55c84 => generated 312 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:11.779538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175d50f-ffb2-4292-870d-54f5c3a57ff9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:11.823743 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:11.876914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175d50f-ffb2-4292-870d-54f5c3a57ff9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:11.910823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c39bb-f88d-4331-8d86-0e76f5343021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:11.911439 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c39bb-f88d-4331-8d86-0e76f5343021 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:11.911725 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c39bb-f88d-4331-8d86-0e76f5343021 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:11.912011 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c39bb-f88d-4331-8d86-0e76f5343021 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:11.912795 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c39bb-f88d-4331-8d86-0e76f5343021 None admin] RBAC: Adding query filter params (role.id=bb986e49c38c40bab629ff32371b9358, scope.project.id=e187ee4c4dfa4496b94a7ffedfc55c84, user.id=2d80ea128f63433597c76804fc999bca) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:11.913121 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c39bb-f88d-4331-8d86-0e76f5343021 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:11.914558 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c39bb-f88d-4331-8d86-0e76f5343021 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:11.919956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 563/1117] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:59:11 2022] GET /identity/v3/role_assignments?user.id=2d80ea128f63433597c76804fc999bca&scope.project.id=e187ee4c4dfa4496b94a7ffedfc55c84&role.id=bb986e49c38c40bab629ff32371b9358 => generated 260 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:13.855694 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed038c-112a-4ea2-9ad1-ed896f99c522 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:13.856241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed038c-112a-4ea2-9ad1-ed896f99c522 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:13.856241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed038c-112a-4ea2-9ad1-ed896f99c522 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:13.857661 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 555/1118] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:59:13.862746 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01530327-682c-4594-9588-6e616129c7ba None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:13.863137 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01530327-682c-4594-9588-6e616129c7ba None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:13.863343 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01530327-682c-4594-9588-6e616129c7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:13.881192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-01530327-682c-4594-9588-6e616129c7ba None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:13.882354 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-01530327-682c-4594-9588-6e616129c7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:13.948851 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 564/1119] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:13 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:59:13.971413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed038c-112a-4ea2-9ad1-ed896f99c522 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:14.068434 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed038c-112a-4ea2-9ad1-ed896f99c522 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:14.100767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405289f9-613d-4ea9-8ff5-a327fe9ed29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:14.101420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405289f9-613d-4ea9-8ff5-a327fe9ed29a None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:14.101705 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405289f9-613d-4ea9-8ff5-a327fe9ed29a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:14.101980 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405289f9-613d-4ea9-8ff5-a327fe9ed29a None admin] PATH_INFO: `/v3/roles/bb986e49c38c40bab629ff32371b9358` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:14.103398 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405289f9-613d-4ea9-8ff5-a327fe9ed29a None admin] RBAC: Authorizing `identity:get_role(role_id=bb986e49c38c40bab629ff32371b9358)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:14.104762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405289f9-613d-4ea9-8ff5-a327fe9ed29a None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:14.106188 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 556/1120] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:59:13 2022] GET /identity/v3/roles/bb986e49c38c40bab629ff32371b9358 => generated 224 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:14.111812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01530327-682c-4594-9588-6e616129c7ba None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:14.150704 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:14.210574 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01530327-682c-4594-9588-6e616129c7ba None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:14.245604 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ff94e-768f-4e82-93b2-b072673d6ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:14.246242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ff94e-768f-4e82-93b2-b072673d6ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:14.246525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ff94e-768f-4e82-93b2-b072673d6ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:14.246803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ff94e-768f-4e82-93b2-b072673d6ce6 None admin] PATH_INFO: `/v3/users/2d80ea128f63433597c76804fc999bca` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:14.247955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ff94e-768f-4e82-93b2-b072673d6ce6 None admin] RBAC: Authorizing `identity:get_user(user_id=2d80ea128f63433597c76804fc999bca)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:14.249659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ff94e-768f-4e82-93b2-b072673d6ce6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:14.251277 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 565/1121] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:59:14 2022] GET /identity/v3/users/2d80ea128f63433597c76804fc999bca => generated 287 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:14.257204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405289f9-613d-4ea9-8ff5-a327fe9ed29a None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:14.268898 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:14.356826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405289f9-613d-4ea9-8ff5-a327fe9ed29a None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:14.390083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e66ef-711e-4595-9ed1-956541b93150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:14.390704 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e66ef-711e-4595-9ed1-956541b93150 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:14.390990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e66ef-711e-4595-9ed1-956541b93150 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:14.391263 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e66ef-711e-4595-9ed1-956541b93150 None admin] PATH_INFO: `/v3/projects/e187ee4c4dfa4496b94a7ffedfc55c84` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:14.392396 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e66ef-711e-4595-9ed1-956541b93150 None admin] RBAC: Authorizing `identity:get_project(project_id=e187ee4c4dfa4496b94a7ffedfc55c84)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:14.393845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e66ef-711e-4595-9ed1-956541b93150 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:14.395939 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 557/1122] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:59:14 2022] GET /identity/v3/projects/e187ee4c4dfa4496b94a7ffedfc55c84 => generated 312 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:14.401872 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ff94e-768f-4e82-93b2-b072673d6ce6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:14.501067 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ff94e-768f-4e82-93b2-b072673d6ce6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:14.508692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:14.536205 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6108f71-9ebb-492e-b023-23e6dfaf3460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:14.536861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6108f71-9ebb-492e-b023-23e6dfaf3460 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:14.537166 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6108f71-9ebb-492e-b023-23e6dfaf3460 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:14.537446 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6108f71-9ebb-492e-b023-23e6dfaf3460 None admin] PATH_INFO: `/v3/projects/e187ee4c4dfa4496b94a7ffedfc55c84/users/2d80ea128f63433597c76804fc999bca/roles/bb986e49c38c40bab629ff32371b9358` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:14.539594 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6108f71-9ebb-492e-b023-23e6dfaf3460 None admin] RBAC: Authorizing `identity:create_grant(project_id=e187ee4c4dfa4496b94a7ffedfc55c84, user_id=2d80ea128f63433597c76804fc999bca, role_id=bb986e49c38c40bab629ff32371b9358)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:14.541003 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6108f71-9ebb-492e-b023-23e6dfaf3460 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:14.553936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 566/1123] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:59:14 2022] PUT /identity/v3/projects/e187ee4c4dfa4496b94a7ffedfc55c84/users/2d80ea128f63433597c76804fc999bca/roles/bb986e49c38c40bab629ff32371b9358 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:59:16.511985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3dba9-953d-4248-9948-a038196992c6 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:16.512535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3dba9-953d-4248-9948-a038196992c6 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:16.512535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3dba9-953d-4248-9948-a038196992c6 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:16.513899 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 558/1124] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:59:16.518938 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117e6b9-7c81-490c-9f10-cac3816fdfca None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:16.519968 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117e6b9-7c81-490c-9f10-cac3816fdfca None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:16.520069 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117e6b9-7c81-490c-9f10-cac3816fdfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:16.537784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-7117e6b9-7c81-490c-9f10-cac3816fdfca None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:16.538963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7117e6b9-7c81-490c-9f10-cac3816fdfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:16.603713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 567/1125] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:16 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:59:16.626129 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3dba9-953d-4248-9948-a038196992c6 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:16.701015 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:16.724215 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3dba9-953d-4248-9948-a038196992c6 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:16.757563 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c369d1a-34e8-436c-a8ed-dec00022742b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:16.758194 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c369d1a-34e8-436c-a8ed-dec00022742b None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:16.758499 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c369d1a-34e8-436c-a8ed-dec00022742b None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:16.758776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c369d1a-34e8-436c-a8ed-dec00022742b None admin] PATH_INFO: `/v3/roles/bb986e49c38c40bab629ff32371b9358` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:16.760202 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c369d1a-34e8-436c-a8ed-dec00022742b None admin] RBAC: Authorizing `identity:get_role(role_id=bb986e49c38c40bab629ff32371b9358)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:16.761599 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c369d1a-34e8-436c-a8ed-dec00022742b None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:16.762981 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 559/1126] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:59:16 2022] GET /identity/v3/roles/bb986e49c38c40bab629ff32371b9358 => generated 224 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:16.768662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117e6b9-7c81-490c-9f10-cac3816fdfca None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:16.860596 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:16.867854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117e6b9-7c81-490c-9f10-cac3816fdfca None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:16.901710 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429dc5f9-b106-4906-9c5b-d7645bb4e167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:16.902355 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dc5f9-b106-4906-9c5b-d7645bb4e167 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:16.902640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dc5f9-b106-4906-9c5b-d7645bb4e167 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:16.902923 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dc5f9-b106-4906-9c5b-d7645bb4e167 None admin] PATH_INFO: `/v3/users/2d80ea128f63433597c76804fc999bca` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:16.904093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429dc5f9-b106-4906-9c5b-d7645bb4e167 None admin] RBAC: Authorizing `identity:get_user(user_id=2d80ea128f63433597c76804fc999bca)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:16.905549 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429dc5f9-b106-4906-9c5b-d7645bb4e167 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:16.907195 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 568/1127] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:59:16 2022] GET /identity/v3/users/2d80ea128f63433597c76804fc999bca => generated 287 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:16.913079 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c369d1a-34e8-436c-a8ed-dec00022742b None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:17.011831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c369d1a-34e8-436c-a8ed-dec00022742b None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:17.044459 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f7207-c22f-4cd4-8565-006623f0f2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:17.045000 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f7207-c22f-4cd4-8565-006623f0f2f7 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:17.045318 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f7207-c22f-4cd4-8565-006623f0f2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:17.045595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f7207-c22f-4cd4-8565-006623f0f2f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:17.060154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:17.112175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f7207-c22f-4cd4-8565-006623f0f2f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:17.113652 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f7207-c22f-4cd4-8565-006623f0f2f7 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:17.207207 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 560/1128] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:59:16 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:59:17.213349 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429dc5f9-b106-4906-9c5b-d7645bb4e167 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:17.309520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429dc5f9-b106-4906-9c5b-d7645bb4e167 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:17.341877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7819c-4137-47ad-bb4c-3415852d3770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:17.342506 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7819c-4137-47ad-bb4c-3415852d3770 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:17.342785 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7819c-4137-47ad-bb4c-3415852d3770 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:17.343062 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7819c-4137-47ad-bb4c-3415852d3770 None admin] PATH_INFO: `/v3/projects/e187ee4c4dfa4496b94a7ffedfc55c84` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:17.344184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7819c-4137-47ad-bb4c-3415852d3770 None admin] RBAC: Authorizing `identity:get_project(project_id=e187ee4c4dfa4496b94a7ffedfc55c84)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:17.345621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7819c-4137-47ad-bb4c-3415852d3770 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:17.347248 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 569/1129] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:59:17 2022] GET /identity/v3/projects/e187ee4c4dfa4496b94a7ffedfc55c84 => generated 312 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:17.354099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f7207-c22f-4cd4-8565-006623f0f2f7 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:17.398810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:17.451745 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f7207-c22f-4cd4-8565-006623f0f2f7 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:17.484619 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763adb90-7b07-4fc4-b0cc-1bbe682294ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:17.485249 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763adb90-7b07-4fc4-b0cc-1bbe682294ca None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:17.485535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763adb90-7b07-4fc4-b0cc-1bbe682294ca None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:17.485808 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763adb90-7b07-4fc4-b0cc-1bbe682294ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:17.486580 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763adb90-7b07-4fc4-b0cc-1bbe682294ca None admin] RBAC: Adding query filter params (role.id=bb986e49c38c40bab629ff32371b9358, scope.project.id=e187ee4c4dfa4496b94a7ffedfc55c84, user.id=2d80ea128f63433597c76804fc999bca) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:17.486891 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763adb90-7b07-4fc4-b0cc-1bbe682294ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:17.488311 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763adb90-7b07-4fc4-b0cc-1bbe682294ca None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:17.494531 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 561/1130] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:59:17 2022] GET /identity/v3/role_assignments?user.id=2d80ea128f63433597c76804fc999bca&scope.project.id=e187ee4c4dfa4496b94a7ffedfc55c84&role.id=bb986e49c38c40bab629ff32371b9358 => generated 610 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:19.495884 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6657b-25df-48c8-a638-ce60869c896b None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:19.496413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6657b-25df-48c8-a638-ce60869c896b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:19.496413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6657b-25df-48c8-a638-ce60869c896b None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:19.497838 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 570/1131] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:59:19.503226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacaf5df-be9b-4c43-8781-03b5861bad10 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:19.503630 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacaf5df-be9b-4c43-8781-03b5861bad10 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:19.503770 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacaf5df-be9b-4c43-8781-03b5861bad10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:19.522196 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-aacaf5df-be9b-4c43-8781-03b5861bad10 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:19.523350 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-aacaf5df-be9b-4c43-8781-03b5861bad10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:19.589345 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 562/1132] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:19 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:59:19.611886 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6657b-25df-48c8-a638-ce60869c896b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:19.642688 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:19.712266 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6657b-25df-48c8-a638-ce60869c896b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:19.745961 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa740ff-294a-4603-8fc9-9401bb7b42bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:19.746543 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa740ff-294a-4603-8fc9-9401bb7b42bb None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:19.746853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa740ff-294a-4603-8fc9-9401bb7b42bb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:19.747133 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa740ff-294a-4603-8fc9-9401bb7b42bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:19.748280 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa740ff-294a-4603-8fc9-9401bb7b42bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:19.749985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa740ff-294a-4603-8fc9-9401bb7b42bb None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:19.751568 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 571/1133] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:59:19 2022] GET /identity/v3/domains/default => generated 200 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:19.757568 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacaf5df-be9b-4c43-8781-03b5861bad10 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:19.796575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:19.855370 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacaf5df-be9b-4c43-8781-03b5861bad10 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:19.888030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62285d1-b1a6-456e-9eb1-602ec5f02eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:19.888633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62285d1-b1a6-456e-9eb1-602ec5f02eb6 None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:19.888919 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62285d1-b1a6-456e-9eb1-602ec5f02eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:19.889226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62285d1-b1a6-456e-9eb1-602ec5f02eb6 None admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:19.889903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62285d1-b1a6-456e-9eb1-602ec5f02eb6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:19.891330 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62285d1-b1a6-456e-9eb1-602ec5f02eb6 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:19.918386 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 563/1134] 10.222.0.100 () {60 vars in 1146 bytes} [Tue Aug 2 08:59:19 2022] POST /identity/v3/projects => generated 312 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:59:21.901181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99da3d-7e56-4959-842e-1ae4ba7993dc None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:21.901737 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99da3d-7e56-4959-842e-1ae4ba7993dc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:21.901737 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99da3d-7e56-4959-842e-1ae4ba7993dc None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:21.903241 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 572/1135] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:59:21.908168 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c66ee-e4f2-491e-9f15-99a7698cf504 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:21.908568 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c66ee-e4f2-491e-9f15-99a7698cf504 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:21.908720 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c66ee-e4f2-491e-9f15-99a7698cf504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:21.928144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-373c66ee-e4f2-491e-9f15-99a7698cf504 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:21.929363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-373c66ee-e4f2-491e-9f15-99a7698cf504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:21.994000 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 564/1136] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:21 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:59:22.017706 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e99da3d-7e56-4959-842e-1ae4ba7993dc None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:22.114282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:22.118491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e99da3d-7e56-4959-842e-1ae4ba7993dc None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:22.153141 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd421cb1-5ab0-4cce-9b6e-d3eb510221c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:22.153713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd421cb1-5ab0-4cce-9b6e-d3eb510221c7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:22.153996 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd421cb1-5ab0-4cce-9b6e-d3eb510221c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:22.154271 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd421cb1-5ab0-4cce-9b6e-d3eb510221c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:22.155419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd421cb1-5ab0-4cce-9b6e-d3eb510221c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:22.156833 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd421cb1-5ab0-4cce-9b6e-d3eb510221c7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:22.158421 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 573/1137] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 08:59:22 2022] GET /identity/v3/domains/default => generated 200 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:22.164124 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c66ee-e4f2-491e-9f15-99a7698cf504 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:22.185214 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:22.262204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c66ee-e4f2-491e-9f15-99a7698cf504 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:22.294607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdf674-851c-4d49-a1c0-30d33eb399ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:22.295211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdf674-851c-4d49-a1c0-30d33eb399ef None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:22.295495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdf674-851c-4d49-a1c0-30d33eb399ef None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:22.295771 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdf674-851c-4d49-a1c0-30d33eb399ef None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:22.296456 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cdf674-851c-4d49-a1c0-30d33eb399ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:22.297910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cdf674-851c-4d49-a1c0-30d33eb399ef None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:22.319618 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 565/1138] 10.222.0.100 () {60 vars in 1137 bytes} [Tue Aug 2 08:59:22 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:59:24.344602 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe1285-2402-4197-a2b8-bfaab7ab870f None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:24.345188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe1285-2402-4197-a2b8-bfaab7ab870f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:24.345188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe1285-2402-4197-a2b8-bfaab7ab870f None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:24.346538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 574/1139] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:59:24.352401 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee2260-9e0c-4104-b0b9-e852d3def555 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:24.352822 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee2260-9e0c-4104-b0b9-e852d3def555 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:24.352962 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee2260-9e0c-4104-b0b9-e852d3def555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:24.370752 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-61ee2260-9e0c-4104-b0b9-e852d3def555 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:24.371934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-61ee2260-9e0c-4104-b0b9-e852d3def555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:24.436949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 566/1140] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:24 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:59:24.459731 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe1285-2402-4197-a2b8-bfaab7ab870f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:24.559061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe1285-2402-4197-a2b8-bfaab7ab870f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:24.591758 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271efdf1-e542-411b-a6a7-4da08bd6b07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:24.592390 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271efdf1-e542-411b-a6a7-4da08bd6b07c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:24.592674 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271efdf1-e542-411b-a6a7-4da08bd6b07c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:24.592948 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271efdf1-e542-411b-a6a7-4da08bd6b07c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:24.603415 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271efdf1-e542-411b-a6a7-4da08bd6b07c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:59:24.608237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-271efdf1-e542-411b-a6a7-4da08bd6b07c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:59:24.608237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:24.608237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:24.608237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:24.608237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:24.608237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:24.608237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:24.608237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:24.608237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:24.608237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:24.608237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:24.608237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:24.608237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:59:24.608237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:59:24.608237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:59:24.608237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:59:24.609435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:59:24.609435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:59:24.609435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:59:24.609435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:59:24.609435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:24.609435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:24.609435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:24.609435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:24.609435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:24.609435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:24.609435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:24.609435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:24.609435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:24.609435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:24.609435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:24.609435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:24.610382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:24.610382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:24.610382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:24.610382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:24.610382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:59:24.610382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:59:24.610382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:59:24.610382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:59:24.610382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:59:24.610382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:59:24.610382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:59:24.610382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:59:24.610382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 575/1141] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:59:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:59:24.615524 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee2260-9e0c-4104-b0b9-e852d3def555 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:24.623909 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:24.713254 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee2260-9e0c-4104-b0b9-e852d3def555 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:24.746708 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744f74e-ee72-4178-b4c0-731ea23b0cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:24.747304 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744f74e-ee72-4178-b4c0-731ea23b0cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:24.747582 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744f74e-ee72-4178-b4c0-731ea23b0cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:24.747851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744f74e-ee72-4178-b4c0-731ea23b0cb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:24.748802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744f74e-ee72-4178-b4c0-731ea23b0cb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:24.749102 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744f74e-ee72-4178-b4c0-731ea23b0cb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:24.750483 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744f74e-ee72-4178-b4c0-731ea23b0cb9 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:24.758275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 567/1142] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:59:24 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:24.764050 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271efdf1-e542-411b-a6a7-4da08bd6b07c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:24.793393 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:24.866566 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271efdf1-e542-411b-a6a7-4da08bd6b07c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:24.899576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ae707-9560-4940-b071-281f54491903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:24.900212 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ae707-9560-4940-b071-281f54491903 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:24.900495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ae707-9560-4940-b071-281f54491903 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:24.900771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ae707-9560-4940-b071-281f54491903 None admin] PATH_INFO: `/v3/users/5319edce74b844b2816969391a7eea11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:24.914469 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ae707-9560-4940-b071-281f54491903 None admin] RBAC: Authorizing `identity:get_user(user_id=5319edce74b844b2816969391a7eea11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:24.915899 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ae707-9560-4940-b071-281f54491903 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:24.917606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 576/1143] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:59:24 2022] GET /identity/v3/users/5319edce74b844b2816969391a7eea11 => generated 287 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:24.923305 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744f74e-ee72-4178-b4c0-731ea23b0cb9 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:24.986809 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:25.020342 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744f74e-ee72-4178-b4c0-731ea23b0cb9 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:25.052340 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9326f-8c91-4c3f-b46a-3c88520524a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:25.052941 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9326f-8c91-4c3f-b46a-3c88520524a2 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:25.053249 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9326f-8c91-4c3f-b46a-3c88520524a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:25.053519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9326f-8c91-4c3f-b46a-3c88520524a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:25.118895 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9326f-8c91-4c3f-b46a-3c88520524a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:25.120277 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9326f-8c91-4c3f-b46a-3c88520524a2 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:25.182997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:25.216143 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 568/1144] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:59:24 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:59:25.222019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ae707-9560-4940-b071-281f54491903 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:25.304475 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:25.320026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ae707-9560-4940-b071-281f54491903 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:25.352987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947c116-fab9-4fdf-9a5a-69f539b38df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:25.353649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947c116-fab9-4fdf-9a5a-69f539b38df3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:25.353936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947c116-fab9-4fdf-9a5a-69f539b38df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:25.354215 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947c116-fab9-4fdf-9a5a-69f539b38df3 None admin] PATH_INFO: `/v3/projects/6fa1fa98d1de49d89c6cfc970a95bea8` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:25.355364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947c116-fab9-4fdf-9a5a-69f539b38df3 None admin] RBAC: Authorizing `identity:get_project(project_id=6fa1fa98d1de49d89c6cfc970a95bea8)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:25.356782 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947c116-fab9-4fdf-9a5a-69f539b38df3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:25.358620 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 577/1145] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:59:25 2022] GET /identity/v3/projects/6fa1fa98d1de49d89c6cfc970a95bea8 => generated 312 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:25.365626 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9326f-8c91-4c3f-b46a-3c88520524a2 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:25.461312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9326f-8c91-4c3f-b46a-3c88520524a2 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:25.494816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d07bd-607b-4af5-a97a-d7d304c66428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:25.495420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d07bd-607b-4af5-a97a-d7d304c66428 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:25.495704 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d07bd-607b-4af5-a97a-d7d304c66428 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:25.495983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d07bd-607b-4af5-a97a-d7d304c66428 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:25.496759 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d07bd-607b-4af5-a97a-d7d304c66428 None admin] RBAC: Adding query filter params (role.id=7cbff51867a7475d837afd51f6c12f11, scope.project.id=6fa1fa98d1de49d89c6cfc970a95bea8, user.id=5319edce74b844b2816969391a7eea11) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:25.497073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d07bd-607b-4af5-a97a-d7d304c66428 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:25.498780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d07bd-607b-4af5-a97a-d7d304c66428 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:25.504564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 569/1146] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:59:25 2022] GET /identity/v3/role_assignments?user.id=5319edce74b844b2816969391a7eea11&scope.project.id=6fa1fa98d1de49d89c6cfc970a95bea8&role.id=7cbff51867a7475d837afd51f6c12f11 => generated 260 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:27.473358 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c6a13-f7de-4c76-ac95-e051c7530d9f None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:27.473885 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c6a13-f7de-4c76-ac95-e051c7530d9f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:27.473885 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c6a13-f7de-4c76-ac95-e051c7530d9f None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:27.475261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 578/1147] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:59:27.480343 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d8e9c-a6b0-469c-9864-91ec6a69443d None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:27.480765 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d8e9c-a6b0-469c-9864-91ec6a69443d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:27.480892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d8e9c-a6b0-469c-9864-91ec6a69443d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:27.499365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-4a3d8e9c-a6b0-469c-9864-91ec6a69443d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:27.500542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-4a3d8e9c-a6b0-469c-9864-91ec6a69443d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:27.526225 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:27.567360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 570/1148] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:27 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:59:27.589447 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c6a13-f7de-4c76-ac95-e051c7530d9f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:27.689604 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c6a13-f7de-4c76-ac95-e051c7530d9f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:27.721915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b3c63-ec9e-47ae-b9d6-8c0693845571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:27.723117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b3c63-ec9e-47ae-b9d6-8c0693845571 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:27.723612 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b3c63-ec9e-47ae-b9d6-8c0693845571 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:27.724020 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b3c63-ec9e-47ae-b9d6-8c0693845571 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:27.733837 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b3c63-ec9e-47ae-b9d6-8c0693845571 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:59:27.739609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-5a3b3c63-ec9e-47ae-b9d6-8c0693845571 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:59:27.739609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:27.739609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:27.739609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:27.739609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:27.739609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:27.739609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:27.739609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:27.739609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:27.739609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:27.739609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:27.739609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:27.739609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:59:27.739609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:59:27.739609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:59:27.739609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:59:27.740722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:59:27.740722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:59:27.740722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:59:27.740722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:59:27.740722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:27.740722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:27.740722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:27.740722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:27.740722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:27.740722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:27.740722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:27.740722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:27.740722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:27.740722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:27.740722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:27.740722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:27.741784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:27.741784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:27.741784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:27.741784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:27.741784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:59:27.741784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:59:27.741784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:59:27.741784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:59:27.741784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:59:27.741784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:59:27.741784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:59:27.741784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:59:27.741784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 579/1149] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:59:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:59:27.746898 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d8e9c-a6b0-469c-9864-91ec6a69443d None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:27.841285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d8e9c-a6b0-469c-9864-91ec6a69443d None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:27.874651 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221111b-9a3a-4ed0-9a1a-1ed3bc7f6e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:27.875257 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221111b-9a3a-4ed0-9a1a-1ed3bc7f6e05 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:27.875539 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221111b-9a3a-4ed0-9a1a-1ed3bc7f6e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:27.875811 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221111b-9a3a-4ed0-9a1a-1ed3bc7f6e05 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:27.876515 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e221111b-9a3a-4ed0-9a1a-1ed3bc7f6e05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:27.876821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e221111b-9a3a-4ed0-9a1a-1ed3bc7f6e05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:27.878207 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e221111b-9a3a-4ed0-9a1a-1ed3bc7f6e05 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:27.886047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 571/1150] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:59:27 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:27.891768 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b3c63-ec9e-47ae-b9d6-8c0693845571 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:27.907921 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:27.992843 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b3c63-ec9e-47ae-b9d6-8c0693845571 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:28.026157 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da70a8-293b-4f94-8f42-ab05b86832ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:28.026799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da70a8-293b-4f94-8f42-ab05b86832ba None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:28.027083 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da70a8-293b-4f94-8f42-ab05b86832ba None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:28.027362 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da70a8-293b-4f94-8f42-ab05b86832ba None admin] PATH_INFO: `/v3/users/5319edce74b844b2816969391a7eea11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:28.028509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da70a8-293b-4f94-8f42-ab05b86832ba None admin] RBAC: Authorizing `identity:get_user(user_id=5319edce74b844b2816969391a7eea11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:28.029957 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da70a8-293b-4f94-8f42-ab05b86832ba None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:28.031551 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 580/1151] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:59:27 2022] GET /identity/v3/users/5319edce74b844b2816969391a7eea11 => generated 287 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:28.037914 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221111b-9a3a-4ed0-9a1a-1ed3bc7f6e05 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:28.063588 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:28.135950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221111b-9a3a-4ed0-9a1a-1ed3bc7f6e05 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:28.168974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e1548-676b-48fd-9f4a-1aea7551080f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:28.169625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e1548-676b-48fd-9f4a-1aea7551080f None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:28.169905 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e1548-676b-48fd-9f4a-1aea7551080f None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:28.170177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e1548-676b-48fd-9f4a-1aea7551080f None admin] PATH_INFO: `/v3/projects/6fa1fa98d1de49d89c6cfc970a95bea8` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:28.171291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e1548-676b-48fd-9f4a-1aea7551080f None admin] RBAC: Authorizing `identity:get_project(project_id=6fa1fa98d1de49d89c6cfc970a95bea8)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:28.172703 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e1548-676b-48fd-9f4a-1aea7551080f None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:28.174366 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 572/1152] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:59:28 2022] GET /identity/v3/projects/6fa1fa98d1de49d89c6cfc970a95bea8 => generated 312 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:28.180387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da70a8-293b-4f94-8f42-ab05b86832ba None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:28.260309 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:28.279121 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da70a8-293b-4f94-8f42-ab05b86832ba None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:28.312816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23c508-6ee6-4589-b8fe-21cc70c1266e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:28.313506 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23c508-6ee6-4589-b8fe-21cc70c1266e None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:28.313791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23c508-6ee6-4589-b8fe-21cc70c1266e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:28.314068 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23c508-6ee6-4589-b8fe-21cc70c1266e None admin] PATH_INFO: `/v3/projects/6fa1fa98d1de49d89c6cfc970a95bea8/users/5319edce74b844b2816969391a7eea11/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:28.315973 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23c508-6ee6-4589-b8fe-21cc70c1266e None admin] RBAC: Authorizing `identity:create_grant(project_id=6fa1fa98d1de49d89c6cfc970a95bea8, user_id=5319edce74b844b2816969391a7eea11, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:28.317674 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23c508-6ee6-4589-b8fe-21cc70c1266e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:28.330874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 581/1153] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:59:28 2022] PUT /identity/v3/projects/6fa1fa98d1de49d89c6cfc970a95bea8/users/5319edce74b844b2816969391a7eea11/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:59:30.311468 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d433f24-edce-43af-8645-ff668bd06394 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:30.312010 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d433f24-edce-43af-8645-ff668bd06394 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:30.312010 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d433f24-edce-43af-8645-ff668bd06394 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:30.313404 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 573/1154] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:59:30.319000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47cfa6-6510-4724-972c-ff3dae3d9ca6 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:30.319425 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47cfa6-6510-4724-972c-ff3dae3d9ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:30.319563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47cfa6-6510-4724-972c-ff3dae3d9ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:30.337433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-af47cfa6-6510-4724-972c-ff3dae3d9ca6 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:30.338622 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-af47cfa6-6510-4724-972c-ff3dae3d9ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:30.403606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 582/1155] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:30 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:59:30.426643 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d433f24-edce-43af-8645-ff668bd06394 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:30.516721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:30.527178 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d433f24-edce-43af-8645-ff668bd06394 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:30.561980 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c074c6c-d2d4-444c-8913-dd1562aebf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:30.562541 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c074c6c-d2d4-444c-8913-dd1562aebf3d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:30.562821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c074c6c-d2d4-444c-8913-dd1562aebf3d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:30.563094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c074c6c-d2d4-444c-8913-dd1562aebf3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:30.572991 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c074c6c-d2d4-444c-8913-dd1562aebf3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:59:30.577893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-4c074c6c-d2d4-444c-8913-dd1562aebf3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:59:30.577893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:30.577893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:30.577893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:30.577893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:30.577893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:30.577893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:30.577893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:30.577893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:30.577893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:30.577893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:30.577893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:30.577893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:59:30.577893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:59:30.577893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:59:30.577893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:59:30.578997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:59:30.578997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:59:30.578997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:59:30.578997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:59:30.578997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:30.578997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:30.578997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:30.578997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:30.578997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:30.578997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:30.578997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:30.578997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:30.578997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:30.578997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:30.578997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:30.578997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:30.580116 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:30.580116 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:30.580116 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:30.580116 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:30.580116 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:59:30.580116 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:59:30.580116 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:59:30.580116 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:59:30.580116 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:59:30.580116 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:59:30.580116 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:59:30.580116 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:59:30.580116 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 574/1156] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:59:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:59:30.585716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47cfa6-6510-4724-972c-ff3dae3d9ca6 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:30.656942 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:30.686338 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47cfa6-6510-4724-972c-ff3dae3d9ca6 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:30.719974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c16b3-c050-458e-92f9-cb8c79fa520e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:30.720596 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c16b3-c050-458e-92f9-cb8c79fa520e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:30.720879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c16b3-c050-458e-92f9-cb8c79fa520e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:30.721174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c16b3-c050-458e-92f9-cb8c79fa520e None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:30.721883 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c16b3-c050-458e-92f9-cb8c79fa520e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:30.722189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c16b3-c050-458e-92f9-cb8c79fa520e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:30.723564 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c16b3-c050-458e-92f9-cb8c79fa520e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:30.731393 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 583/1157] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:59:30 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:30.737182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c074c6c-d2d4-444c-8913-dd1562aebf3d None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:30.856356 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c074c6c-d2d4-444c-8913-dd1562aebf3d None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:30.896343 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47c37e-78f2-4229-80a1-28c6749231c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:30.896985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47c37e-78f2-4229-80a1-28c6749231c3 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:30.897303 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47c37e-78f2-4229-80a1-28c6749231c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:30.897583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47c37e-78f2-4229-80a1-28c6749231c3 None admin] PATH_INFO: `/v3/users/5319edce74b844b2816969391a7eea11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:30.898765 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c47c37e-78f2-4229-80a1-28c6749231c3 None admin] RBAC: Authorizing `identity:get_user(user_id=5319edce74b844b2816969391a7eea11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:30.900203 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c47c37e-78f2-4229-80a1-28c6749231c3 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:30.902060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 575/1158] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:59:30 2022] GET /identity/v3/users/5319edce74b844b2816969391a7eea11 => generated 287 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:30.908042 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c16b3-c050-458e-92f9-cb8c79fa520e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:31.024961 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c16b3-c050-458e-92f9-cb8c79fa520e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:31.063618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a589b7c-6c23-4403-81ab-6ea5d3fa0a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:31.064248 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a589b7c-6c23-4403-81ab-6ea5d3fa0a58 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:31.064533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a589b7c-6c23-4403-81ab-6ea5d3fa0a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:31.064813 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a589b7c-6c23-4403-81ab-6ea5d3fa0a58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:31.068237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:31.139475 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a589b7c-6c23-4403-81ab-6ea5d3fa0a58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:31.140925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a589b7c-6c23-4403-81ab-6ea5d3fa0a58 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:31.244863 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 584/1159] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:59:30 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:59:31.251132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47c37e-78f2-4229-80a1-28c6749231c3 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:31.276174 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:31.359647 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47c37e-78f2-4229-80a1-28c6749231c3 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:31.397057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647c5fa-5a34-46aa-8612-66e2ec2480ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:31.397651 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647c5fa-5a34-46aa-8612-66e2ec2480ba None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:31.397936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647c5fa-5a34-46aa-8612-66e2ec2480ba None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:31.398217 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647c5fa-5a34-46aa-8612-66e2ec2480ba None admin] PATH_INFO: `/v3/projects/6fa1fa98d1de49d89c6cfc970a95bea8` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:31.399373 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647c5fa-5a34-46aa-8612-66e2ec2480ba None admin] RBAC: Authorizing `identity:get_project(project_id=6fa1fa98d1de49d89c6cfc970a95bea8)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:31.400819 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647c5fa-5a34-46aa-8612-66e2ec2480ba None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:31.402529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 576/1160] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:59:31 2022] GET /identity/v3/projects/6fa1fa98d1de49d89c6cfc970a95bea8 => generated 312 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:31.408768 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a589b7c-6c23-4403-81ab-6ea5d3fa0a58 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:31.447656 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:31.514675 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a589b7c-6c23-4403-81ab-6ea5d3fa0a58 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:31.548136 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9332a-6bc3-456e-892c-34b110213902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:31.548751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9332a-6bc3-456e-892c-34b110213902 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:31.549034 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9332a-6bc3-456e-892c-34b110213902 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:31.549368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9332a-6bc3-456e-892c-34b110213902 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:31.550148 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9332a-6bc3-456e-892c-34b110213902 None admin] RBAC: Adding query filter params (role.id=7cbff51867a7475d837afd51f6c12f11, scope.project.id=6fa1fa98d1de49d89c6cfc970a95bea8, user.id=5319edce74b844b2816969391a7eea11) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:31.550536 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9332a-6bc3-456e-892c-34b110213902 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:31.552052 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9332a-6bc3-456e-892c-34b110213902 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:31.557812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 585/1161] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:59:31 2022] GET /identity/v3/role_assignments?user.id=5319edce74b844b2816969391a7eea11&scope.project.id=6fa1fa98d1de49d89c6cfc970a95bea8&role.id=7cbff51867a7475d837afd51f6c12f11 => generated 610 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:33.540379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770e7b5-aebb-476b-b365-54436b131191 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:33.540973 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770e7b5-aebb-476b-b365-54436b131191 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:33.540973 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770e7b5-aebb-476b-b365-54436b131191 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:33.542324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 577/1162] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:59:33.547394 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be13abe-04d8-4e09-bdf6-5e6e2605870a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:33.547850 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be13abe-04d8-4e09-bdf6-5e6e2605870a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:33.547949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be13abe-04d8-4e09-bdf6-5e6e2605870a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:33.565638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5be13abe-04d8-4e09-bdf6-5e6e2605870a None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:33.566955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5be13abe-04d8-4e09-bdf6-5e6e2605870a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:33.632657 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 586/1163] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:33 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:59:33.655041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770e7b5-aebb-476b-b365-54436b131191 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:33.741644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:33.755214 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770e7b5-aebb-476b-b365-54436b131191 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:33.788500 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cdaf5-383d-40c9-a795-926e4fba348b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:33.789098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cdaf5-383d-40c9-a795-926e4fba348b None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:33.789426 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cdaf5-383d-40c9-a795-926e4fba348b None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:33.789701 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cdaf5-383d-40c9-a795-926e4fba348b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:33.854484 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cdaf5-383d-40c9-a795-926e4fba348b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:33.855861 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cdaf5-383d-40c9-a795-926e4fba348b None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:33.942386 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:33.955088 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 578/1164] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:59:33 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:59:33.961673 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be13abe-04d8-4e09-bdf6-5e6e2605870a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:33.989751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:34.061052 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be13abe-04d8-4e09-bdf6-5e6e2605870a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:34.094669 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2459fab-93f8-4759-9190-dfab4544b0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:34.095300 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2459fab-93f8-4759-9190-dfab4544b0d9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:34.095581 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2459fab-93f8-4759-9190-dfab4544b0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:34.095852 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2459fab-93f8-4759-9190-dfab4544b0d9 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:34.102793 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2459fab-93f8-4759-9190-dfab4544b0d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:34.104209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2459fab-93f8-4759-9190-dfab4544b0d9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:34.116458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-f2459fab-93f8-4759-9190-dfab4544b0d9 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 02 08:59:34.116458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:34.116458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 02 08:59:34.116458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 02 08:59:34.116458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:59:34.116458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:59:34.116458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:59:34.116458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:59:34.116458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 02 08:59:34.116458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:59:34.116458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:59:34.116458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:59:34.116458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:34.116458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:34.116458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:34.116458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:34.116458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:34.117667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:34.117667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:34.117667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:34.117667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:34.117667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:34.117667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:34.117667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:59:34.117667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:59:34.117667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:59:34.117667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:59:34.117667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:34.117667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:34.117667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:34.117667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:34.117667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:34.117667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:34.118925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:34.118925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:34.118925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:34.118925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:34.118925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:34.118925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:34.118925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:34.118925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:34.118925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:34.118925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:34.118925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 02 08:59:34.118925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:59:34.118925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 02 08:59:34.118925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:59:34.118925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 587/1165] 10.222.0.100 () {56 vars in 1122 bytes} [Tue Aug 2 08:59:33 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:59:34.123755 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cdaf5-383d-40c9-a795-926e4fba348b None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:34.218502 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cdaf5-383d-40c9-a795-926e4fba348b None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:34.251177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69aae1-38a1-4bee-9ac8-a91d6864083f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:34.251773 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69aae1-38a1-4bee-9ac8-a91d6864083f None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:34.252052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69aae1-38a1-4bee-9ac8-a91d6864083f None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:34.252324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69aae1-38a1-4bee-9ac8-a91d6864083f None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:34.252998 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69aae1-38a1-4bee-9ac8-a91d6864083f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:34.253896 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69aae1-38a1-4bee-9ac8-a91d6864083f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:34.255241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69aae1-38a1-4bee-9ac8-a91d6864083f None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:34.262123 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 579/1166] 10.222.0.100 () {56 vars in 1120 bytes} [Tue Aug 2 08:59:34 2022] GET /identity/v3/domains?name=swift_test => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:36.281119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53571b72-e702-430a-85e7-59a91d3532fd None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:36.281624 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53571b72-e702-430a-85e7-59a91d3532fd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:36.281741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53571b72-e702-430a-85e7-59a91d3532fd None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:36.283069 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 588/1167] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:59:36.288428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77625a87-b7fd-4319-9a1e-0baaea3c14df None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:36.288830 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77625a87-b7fd-4319-9a1e-0baaea3c14df None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:36.288991 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77625a87-b7fd-4319-9a1e-0baaea3c14df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:36.307142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-77625a87-b7fd-4319-9a1e-0baaea3c14df None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:36.308318 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-77625a87-b7fd-4319-9a1e-0baaea3c14df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:36.353453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:36.374320 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 580/1168] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:36 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:59:36.397346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53571b72-e702-430a-85e7-59a91d3532fd None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:36.482987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:36.496106 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53571b72-e702-430a-85e7-59a91d3532fd None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:36.529506 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24b3a3-f66f-4185-ae54-c003ab47914b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:36.530118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24b3a3-f66f-4185-ae54-c003ab47914b None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:36.530402 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24b3a3-f66f-4185-ae54-c003ab47914b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:36.530679 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24b3a3-f66f-4185-ae54-c003ab47914b None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:36.531438 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24b3a3-f66f-4185-ae54-c003ab47914b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:36.532748 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24b3a3-f66f-4185-ae54-c003ab47914b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:36.548229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 589/1169] 10.222.0.100 () {60 vars in 1143 bytes} [Tue Aug 2 08:59:36 2022] POST /identity/v3/domains => generated 268 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:59:38.588915 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2a49c-184a-41ff-adaa-9f645c662556 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:38.589518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2a49c-184a-41ff-adaa-9f645c662556 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:38.589518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2a49c-184a-41ff-adaa-9f645c662556 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:38.590881 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 581/1170] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:59:38.595980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a2a4b-8b44-469f-8cbb-b07f9262bf58 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:38.596407 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a2a4b-8b44-469f-8cbb-b07f9262bf58 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:38.596546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a2a4b-8b44-469f-8cbb-b07f9262bf58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:38.615171 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-822a2a4b-8b44-469f-8cbb-b07f9262bf58 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:38.616360 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-822a2a4b-8b44-469f-8cbb-b07f9262bf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:38.684013 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 590/1171] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:38 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:59:38.707287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2a49c-184a-41ff-adaa-9f645c662556 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:38.806692 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2a49c-184a-41ff-adaa-9f645c662556 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:38.839895 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03b159-febe-4bab-aa61-1b951abd96a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:38.840458 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03b159-febe-4bab-aa61-1b951abd96a4 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:38.840741 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03b159-febe-4bab-aa61-1b951abd96a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:38.841019 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03b159-febe-4bab-aa61-1b951abd96a4 None admin] PATH_INFO: `/v3/domains/a993772696cd41e78149515d67338ee8` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:38.851882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03b159-febe-4bab-aa61-1b951abd96a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a993772696cd41e78149515d67338ee8)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:38.853360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03b159-febe-4bab-aa61-1b951abd96a4 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:38.854956 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 582/1172] 10.222.0.100 () {56 vars in 1188 bytes} [Tue Aug 2 08:59:38 2022] GET /identity/v3/domains/a993772696cd41e78149515d67338ee8 => generated 268 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:38.860765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a2a4b-8b44-469f-8cbb-b07f9262bf58 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:38.941849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:38.963998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a2a4b-8b44-469f-8cbb-b07f9262bf58 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:38.998630 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893462ee-763c-4276-8d44-fd72c57a0933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:38.999242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893462ee-763c-4276-8d44-fd72c57a0933 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:38.999525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893462ee-763c-4276-8d44-fd72c57a0933 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:38.999800 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893462ee-763c-4276-8d44-fd72c57a0933 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:39.000473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893462ee-763c-4276-8d44-fd72c57a0933 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:39.001916 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893462ee-763c-4276-8d44-fd72c57a0933 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:39.027455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 591/1173] 10.222.0.100 () {60 vars in 1146 bytes} [Tue Aug 2 08:59:38 2022] POST /identity/v3/projects => generated 362 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:59:41.091995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201dca6-9fc9-430a-9328-39f07caf5aa6 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:41.092517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201dca6-9fc9-430a-9328-39f07caf5aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:41.092607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201dca6-9fc9-430a-9328-39f07caf5aa6 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:41.093934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 583/1174] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:59:41.099316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee497b-ea9e-4167-a29a-a34c7dbeb5ef None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:41.099715 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee497b-ea9e-4167-a29a-a34c7dbeb5ef None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:41.099889 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee497b-ea9e-4167-a29a-a34c7dbeb5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:41.117549 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-61ee497b-ea9e-4167-a29a-a34c7dbeb5ef None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:41.118711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-61ee497b-ea9e-4167-a29a-a34c7dbeb5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:41.182525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 592/1175] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:41 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:59:41.206504 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201dca6-9fc9-430a-9328-39f07caf5aa6 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:41.250444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:41.306826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201dca6-9fc9-430a-9328-39f07caf5aa6 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:41.340230 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7f79f-4089-49d1-a799-17311a295f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:41.340882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7f79f-4089-49d1-a799-17311a295f11 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:41.341300 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7f79f-4089-49d1-a799-17311a295f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:41.341590 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7f79f-4089-49d1-a799-17311a295f11 None admin] PATH_INFO: `/v3/domains/a993772696cd41e78149515d67338ee8` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:41.342734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7f79f-4089-49d1-a799-17311a295f11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a993772696cd41e78149515d67338ee8)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:41.344163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7f79f-4089-49d1-a799-17311a295f11 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:41.345765 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 584/1176] 10.222.0.100 () {56 vars in 1188 bytes} [Tue Aug 2 08:59:41 2022] GET /identity/v3/domains/a993772696cd41e78149515d67338ee8 => generated 268 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:41.352065 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee497b-ea9e-4167-a29a-a34c7dbeb5ef None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:41.409402 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:41.449136 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee497b-ea9e-4167-a29a-a34c7dbeb5ef None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:41.483553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ebebe-9dfd-4224-b8f4-7fd5131747e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:41.484168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ebebe-9dfd-4224-b8f4-7fd5131747e5 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:41.484458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ebebe-9dfd-4224-b8f4-7fd5131747e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:41.484736 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ebebe-9dfd-4224-b8f4-7fd5131747e5 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:41.485457 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9ebebe-9dfd-4224-b8f4-7fd5131747e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:41.486875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9ebebe-9dfd-4224-b8f4-7fd5131747e5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:41.508180 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 593/1177] 10.222.0.100 () {60 vars in 1137 bytes} [Tue Aug 2 08:59:41 2022] POST /identity/v3/users => generated 312 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:59:43.515135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fabee9-9557-4070-b10d-559f69eab268 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:43.515657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fabee9-9557-4070-b10d-559f69eab268 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:43.515748 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fabee9-9557-4070-b10d-559f69eab268 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:43.517046 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 585/1178] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:59:43.522762 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f76ae-04f9-4555-a0ae-cb345f2163b4 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:43.523199 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f76ae-04f9-4555-a0ae-cb345f2163b4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:43.523318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f76ae-04f9-4555-a0ae-cb345f2163b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:43.540890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-295f76ae-04f9-4555-a0ae-cb345f2163b4 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:43.542087 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-295f76ae-04f9-4555-a0ae-cb345f2163b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:43.606276 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 594/1179] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:43 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:59:43.630040 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fabee9-9557-4070-b10d-559f69eab268 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:43.727948 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fabee9-9557-4070-b10d-559f69eab268 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:43.738887 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:43.761550 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ceea3-2f61-446f-bbd7-9fbdc14700aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:43.762177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ceea3-2f61-446f-bbd7-9fbdc14700aa None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:43.762454 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ceea3-2f61-446f-bbd7-9fbdc14700aa None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:43.762723 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ceea3-2f61-446f-bbd7-9fbdc14700aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:43.772989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ceea3-2f61-446f-bbd7-9fbdc14700aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:59:43.777786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-9a4ceea3-2f61-446f-bbd7-9fbdc14700aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:59:43.777786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:43.777786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:43.777786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:43.777786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:43.777786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:43.777786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:43.777786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:43.777786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:43.777786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:43.777786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:43.777786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:43.777786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:59:43.777786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:59:43.777786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:59:43.777786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:59:43.778715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:59:43.778715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:59:43.778715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:59:43.778715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:59:43.778715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:43.778715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:43.778715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:43.778715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:43.778715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:43.778715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:43.778715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:43.778715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:43.778715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:43.778715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:43.778715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:43.778715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:43.779593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:43.779593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:43.779593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:43.779593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:43.779593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:59:43.779593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:59:43.779593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:59:43.779593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:59:43.779593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:59:43.779593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:59:43.779593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:59:43.779593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:59:43.779593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 586/1180] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:59:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:59:43.785724 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f76ae-04f9-4555-a0ae-cb345f2163b4 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:43.830546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:43.883745 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f76ae-04f9-4555-a0ae-cb345f2163b4 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:43.919064 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b84d53-a127-49e9-a0e7-85e6eb3cc119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:43.919696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b84d53-a127-49e9-a0e7-85e6eb3cc119 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:43.919982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b84d53-a127-49e9-a0e7-85e6eb3cc119 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:43.920261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b84d53-a127-49e9-a0e7-85e6eb3cc119 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:43.920982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b84d53-a127-49e9-a0e7-85e6eb3cc119 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:43.921331 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b84d53-a127-49e9-a0e7-85e6eb3cc119 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:43.922723 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b84d53-a127-49e9-a0e7-85e6eb3cc119 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:43.930937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 595/1181] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:59:43 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:59:43.936669 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ceea3-2f61-446f-bbd7-9fbdc14700aa None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:44.032073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ceea3-2f61-446f-bbd7-9fbdc14700aa None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:44.064862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01205bec-a277-4355-9490-043007f79579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:44.065540 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01205bec-a277-4355-9490-043007f79579 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:44.065905 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01205bec-a277-4355-9490-043007f79579 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:44.066189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01205bec-a277-4355-9490-043007f79579 None admin] PATH_INFO: `/v3/users/cac80d51101a44cc91e73835ff0305a1` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:44.078892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01205bec-a277-4355-9490-043007f79579 None admin] RBAC: Authorizing `identity:get_user(user_id=cac80d51101a44cc91e73835ff0305a1)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:44.080336 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01205bec-a277-4355-9490-043007f79579 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:44.082637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 587/1182] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:59:43 2022] GET /identity/v3/users/cac80d51101a44cc91e73835ff0305a1 => generated 312 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:44.088239 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b84d53-a127-49e9-a0e7-85e6eb3cc119 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:44.186527 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b84d53-a127-49e9-a0e7-85e6eb3cc119 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:44.195671 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:44.220618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be50839-88f3-4847-9d16-0e453dc97631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:44.221269 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be50839-88f3-4847-9d16-0e453dc97631 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:44.221613 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be50839-88f3-4847-9d16-0e453dc97631 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:44.221964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be50839-88f3-4847-9d16-0e453dc97631 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:44.287343 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be50839-88f3-4847-9d16-0e453dc97631 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:44.288706 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be50839-88f3-4847-9d16-0e453dc97631 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:44.385706 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 596/1183] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:59:44 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:59:44.391550 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01205bec-a277-4355-9490-043007f79579 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:44.419563 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:44.489265 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01205bec-a277-4355-9490-043007f79579 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:44.522806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3291fa-3a43-40f8-a867-147964171994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:44.523432 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3291fa-3a43-40f8-a867-147964171994 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:44.523714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3291fa-3a43-40f8-a867-147964171994 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:44.523988 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3291fa-3a43-40f8-a867-147964171994 None admin] PATH_INFO: `/v3/projects/968d6c44427c4579bb79f6c800830669` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:44.525148 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3291fa-3a43-40f8-a867-147964171994 None admin] RBAC: Authorizing `identity:get_project(project_id=968d6c44427c4579bb79f6c800830669)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:44.526576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3291fa-3a43-40f8-a867-147964171994 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:44.528214 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 588/1184] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:59:44 2022] GET /identity/v3/projects/968d6c44427c4579bb79f6c800830669 => generated 362 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:44.534307 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be50839-88f3-4847-9d16-0e453dc97631 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:44.544227 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:44.630430 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be50839-88f3-4847-9d16-0e453dc97631 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:44.663253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a0394-deb9-402b-86d5-a7e605fc4861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:44.663852 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a0394-deb9-402b-86d5-a7e605fc4861 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:44.664131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a0394-deb9-402b-86d5-a7e605fc4861 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:44.664401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a0394-deb9-402b-86d5-a7e605fc4861 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:44.665202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a0394-deb9-402b-86d5-a7e605fc4861 None admin] RBAC: Adding query filter params (role.id=7cbff51867a7475d837afd51f6c12f11, scope.project.id=968d6c44427c4579bb79f6c800830669, user.id=cac80d51101a44cc91e73835ff0305a1) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:44.665510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a0394-deb9-402b-86d5-a7e605fc4861 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:44.666908 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a0394-deb9-402b-86d5-a7e605fc4861 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:44.672111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 597/1185] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:59:44 2022] GET /identity/v3/role_assignments?user.id=cac80d51101a44cc91e73835ff0305a1&scope.project.id=968d6c44427c4579bb79f6c800830669&role.id=7cbff51867a7475d837afd51f6c12f11 => generated 260 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:46.646559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d103c-7e96-4d14-bd0a-3acadb82e464 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:46.647137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d103c-7e96-4d14-bd0a-3acadb82e464 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:46.647137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d103c-7e96-4d14-bd0a-3acadb82e464 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:46.648467 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 589/1186] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:59:46.653744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b4ec3-721f-456a-9697-241430814d27 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:46.654141 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b4ec3-721f-456a-9697-241430814d27 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:46.654297 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b4ec3-721f-456a-9697-241430814d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:46.672416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b14b4ec3-721f-456a-9697-241430814d27 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:46.673615 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b14b4ec3-721f-456a-9697-241430814d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:46.738932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 598/1187] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:46 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:59:46.760677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d103c-7e96-4d14-bd0a-3acadb82e464 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:46.852933 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:46.859989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d103c-7e96-4d14-bd0a-3acadb82e464 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:46.893617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67df30-2afc-4352-a0d0-94ebfe5bde59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:46.894514 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67df30-2afc-4352-a0d0-94ebfe5bde59 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:46.894801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67df30-2afc-4352-a0d0-94ebfe5bde59 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:46.895079 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67df30-2afc-4352-a0d0-94ebfe5bde59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:46.905191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa67df30-2afc-4352-a0d0-94ebfe5bde59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:59:46.910293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-aa67df30-2afc-4352-a0d0-94ebfe5bde59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:59:46.910293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:46.910293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:46.910293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:46.910293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:46.910293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:46.910293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:46.910293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:46.910293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:46.910293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:46.910293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:46.910293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:46.910293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:59:46.910293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:59:46.910293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:59:46.910293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:59:46.911460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:59:46.911460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:59:46.911460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:59:46.911460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:59:46.911460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:46.911460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:46.911460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:46.911460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:46.911460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:46.911460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:46.911460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:46.911460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:46.911460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:46.911460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:46.911460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:46.911460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:46.912573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:46.912573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:46.912573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:46.912573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:46.912573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:59:46.912573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:59:46.912573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:59:46.912573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:59:46.912573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:59:46.912573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:59:46.912573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:59:46.912573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 08:59:46.912573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 590/1188] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:59:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:59:46.917519 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b4ec3-721f-456a-9697-241430814d27 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:46.919836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:47.025541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b4ec3-721f-456a-9697-241430814d27 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:47.058456 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013c75f-76e5-426e-ad32-d00e22522e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:47.059065 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013c75f-76e5-426e-ad32-d00e22522e7f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:47.059344 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013c75f-76e5-426e-ad32-d00e22522e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:47.059616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013c75f-76e5-426e-ad32-d00e22522e7f None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:47.060319 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6013c75f-76e5-426e-ad32-d00e22522e7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:47.060623 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6013c75f-76e5-426e-ad32-d00e22522e7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:47.061996 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6013c75f-76e5-426e-ad32-d00e22522e7f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:47.070437 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 599/1189] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:59:46 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:47.076292 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67df30-2afc-4352-a0d0-94ebfe5bde59 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:47.173935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67df30-2afc-4352-a0d0-94ebfe5bde59 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:47.206508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c68ff-4e74-4994-9aac-cc59ac6d1721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:47.207120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c68ff-4e74-4994-9aac-cc59ac6d1721 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:47.207407 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c68ff-4e74-4994-9aac-cc59ac6d1721 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:47.207682 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c68ff-4e74-4994-9aac-cc59ac6d1721 None admin] PATH_INFO: `/v3/users/cac80d51101a44cc91e73835ff0305a1` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:47.208816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c68ff-4e74-4994-9aac-cc59ac6d1721 None admin] RBAC: Authorizing `identity:get_user(user_id=cac80d51101a44cc91e73835ff0305a1)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:47.210272 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c68ff-4e74-4994-9aac-cc59ac6d1721 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:47.211887 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 591/1190] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:59:47 2022] GET /identity/v3/users/cac80d51101a44cc91e73835ff0305a1 => generated 312 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:47.217557 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013c75f-76e5-426e-ad32-d00e22522e7f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:47.280293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:47.317988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013c75f-76e5-426e-ad32-d00e22522e7f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:47.350921 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecb2a4-331e-4fce-97c9-a99fb51af372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:47.351554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecb2a4-331e-4fce-97c9-a99fb51af372 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:47.351835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecb2a4-331e-4fce-97c9-a99fb51af372 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:47.352111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecb2a4-331e-4fce-97c9-a99fb51af372 None admin] PATH_INFO: `/v3/projects/968d6c44427c4579bb79f6c800830669` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:47.353279 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecb2a4-331e-4fce-97c9-a99fb51af372 None admin] RBAC: Authorizing `identity:get_project(project_id=968d6c44427c4579bb79f6c800830669)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:47.354694 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecb2a4-331e-4fce-97c9-a99fb51af372 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:47.356346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 600/1191] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:59:47 2022] GET /identity/v3/projects/968d6c44427c4579bb79f6c800830669 => generated 362 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:47.362342 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c68ff-4e74-4994-9aac-cc59ac6d1721 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:47.388001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:47.461181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c68ff-4e74-4994-9aac-cc59ac6d1721 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:47.493101 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa94436-0cf4-4704-a095-1953da678dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:47.493775 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa94436-0cf4-4704-a095-1953da678dbd None admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:47.494052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa94436-0cf4-4704-a095-1953da678dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:47.494322 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa94436-0cf4-4704-a095-1953da678dbd None admin] PATH_INFO: `/v3/projects/968d6c44427c4579bb79f6c800830669/users/cac80d51101a44cc91e73835ff0305a1/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:47.496192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa94436-0cf4-4704-a095-1953da678dbd None admin] RBAC: Authorizing `identity:create_grant(project_id=968d6c44427c4579bb79f6c800830669, user_id=cac80d51101a44cc91e73835ff0305a1, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:47.497641 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa94436-0cf4-4704-a095-1953da678dbd None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:47.510504 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 592/1192] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 08:59:47 2022] PUT /identity/v3/projects/968d6c44427c4579bb79f6c800830669/users/cac80d51101a44cc91e73835ff0305a1/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:59:49.470827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8611d-4f5b-4f6f-bf7b-7e126d23ea8b None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:49.471876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8611d-4f5b-4f6f-bf7b-7e126d23ea8b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:49.472109 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8611d-4f5b-4f6f-bf7b-7e126d23ea8b None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:49.473591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 601/1193] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 08:59:49 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 08:59:49.478694 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfe490-3ff5-4730-a0f0-2e6267468eb4 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:49.479105 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfe490-3ff5-4730-a0f0-2e6267468eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:49.479248 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfe490-3ff5-4730-a0f0-2e6267468eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:49.497445 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-b0bfe490-3ff5-4730-a0f0-2e6267468eb4 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:49.498871 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b0bfe490-3ff5-4730-a0f0-2e6267468eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:49.562817 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 593/1194] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 08:59:49 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:59:49.585785 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8611d-4f5b-4f6f-bf7b-7e126d23ea8b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:49.687398 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8611d-4f5b-4f6f-bf7b-7e126d23ea8b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:49.715091 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:49.720383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59839dfa-f0a3-435a-9776-edc9ba2bd93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:49.721657 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59839dfa-f0a3-435a-9776-edc9ba2bd93b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:49.721955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59839dfa-f0a3-435a-9776-edc9ba2bd93b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:49.722335 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59839dfa-f0a3-435a-9776-edc9ba2bd93b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:49.733045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59839dfa-f0a3-435a-9776-edc9ba2bd93b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:59:49.738668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-59839dfa-f0a3-435a-9776-edc9ba2bd93b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:59:49.738668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:59:49.738668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:59:49.738668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:59:49.738668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:59:49.738668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:59:49.738668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:59:49.738668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:59:49.738668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:59:49.738668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:59:49.738668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:59:49.738668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:59:49.738668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:59:49.738668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:59:49.738668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:59:49.738668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:59:49.739771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:59:49.739771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:59:49.739771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:59:49.739771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:59:49.739771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:59:49.739771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:59:49.739771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:59:49.739771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:59:49.739771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:59:49.739771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:59:49.739771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:59:49.739771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 08:59:49.739771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:59:49.739771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:59:49.739771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:59:49.739771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:59:49.740798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:59:49.740798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:59:49.740798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:59:49.740798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:59:49.740798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:59:49.740798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:59:49.740798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:59:49.740798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:59:49.740798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:59:49.740798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:59:49.740798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:59:49.740798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 08:59:49.740798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 602/1195] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 08:59:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:59:49.745920 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bfe490-3ff5-4730-a0f0-2e6267468eb4 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:49.760675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:49.843359 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bfe490-3ff5-4730-a0f0-2e6267468eb4 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:49.876474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949411a-8579-4657-b7a1-ead08d2bff9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:49.877084 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949411a-8579-4657-b7a1-ead08d2bff9e None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:49.877430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949411a-8579-4657-b7a1-ead08d2bff9e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:49.877708 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949411a-8579-4657-b7a1-ead08d2bff9e None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:49.878414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949411a-8579-4657-b7a1-ead08d2bff9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:49.878719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949411a-8579-4657-b7a1-ead08d2bff9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:49.880084 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949411a-8579-4657-b7a1-ead08d2bff9e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:49.887876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 594/1196] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 08:59:49 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:49.894084 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59839dfa-f0a3-435a-9776-edc9ba2bd93b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:49.992553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59839dfa-f0a3-435a-9776-edc9ba2bd93b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:50.025604 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3bc1f-be51-4cdd-a868-940f665079c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:50.026241 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3bc1f-be51-4cdd-a868-940f665079c0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:50.026522 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3bc1f-be51-4cdd-a868-940f665079c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:50.026797 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3bc1f-be51-4cdd-a868-940f665079c0 None admin] PATH_INFO: `/v3/users/cac80d51101a44cc91e73835ff0305a1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:50.027942 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c3bc1f-be51-4cdd-a868-940f665079c0 None admin] RBAC: Authorizing `identity:get_user(user_id=cac80d51101a44cc91e73835ff0305a1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:50.029380 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c3bc1f-be51-4cdd-a868-940f665079c0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:50.030963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 603/1197] 10.222.0.100 () {56 vars in 1182 bytes} [Tue Aug 2 08:59:49 2022] GET /identity/v3/users/cac80d51101a44cc91e73835ff0305a1 => generated 312 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:50.036732 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949411a-8579-4657-b7a1-ead08d2bff9e None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:50.115383 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:50.138077 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949411a-8579-4657-b7a1-ead08d2bff9e None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:50.170766 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9ed12-5ebb-4cc4-8eed-2fc3dcc6ef0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:50.171372 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9ed12-5ebb-4cc4-8eed-2fc3dcc6ef0e None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:50.171655 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9ed12-5ebb-4cc4-8eed-2fc3dcc6ef0e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:50.171932 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9ed12-5ebb-4cc4-8eed-2fc3dcc6ef0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:50.237574 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9ed12-5ebb-4cc4-8eed-2fc3dcc6ef0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:50.239005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9ed12-5ebb-4cc4-8eed-2fc3dcc6ef0e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:50.311689 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:50.336009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 595/1198] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 08:59:50 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:59:50.341847 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3bc1f-be51-4cdd-a868-940f665079c0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:50.399954 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:50.440473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3bc1f-be51-4cdd-a868-940f665079c0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:50.473125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d33f7b-977c-4e7e-b784-0f8d1ed6a1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:50.473784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d33f7b-977c-4e7e-b784-0f8d1ed6a1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:50.474066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d33f7b-977c-4e7e-b784-0f8d1ed6a1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:50.474342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d33f7b-977c-4e7e-b784-0f8d1ed6a1b3 None admin] PATH_INFO: `/v3/projects/968d6c44427c4579bb79f6c800830669` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:50.475489 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d33f7b-977c-4e7e-b784-0f8d1ed6a1b3 None admin] RBAC: Authorizing `identity:get_project(project_id=968d6c44427c4579bb79f6c800830669)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:50.476902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d33f7b-977c-4e7e-b784-0f8d1ed6a1b3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:50.478613 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 604/1199] 10.222.0.100 () {56 vars in 1191 bytes} [Tue Aug 2 08:59:50 2022] GET /identity/v3/projects/968d6c44427c4579bb79f6c800830669 => generated 362 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:59:50.484742 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9ed12-5ebb-4cc4-8eed-2fc3dcc6ef0e None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:50.582086 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9ed12-5ebb-4cc4-8eed-2fc3dcc6ef0e None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:50.614969 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4333c-d091-4775-9757-fb5de919107a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:50.615571 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4333c-d091-4775-9757-fb5de919107a None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:50.615853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4333c-d091-4775-9757-fb5de919107a None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:50.616127 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4333c-d091-4775-9757-fb5de919107a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:50.616897 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4333c-d091-4775-9757-fb5de919107a None admin] RBAC: Adding query filter params (role.id=7cbff51867a7475d837afd51f6c12f11, scope.project.id=968d6c44427c4579bb79f6c800830669, user.id=cac80d51101a44cc91e73835ff0305a1) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:59:50.617233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4333c-d091-4775-9757-fb5de919107a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:50.618655 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4333c-d091-4775-9757-fb5de919107a None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:50.624837 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 596/1200] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 08:59:50 2022] GET /identity/v3/role_assignments?user.id=cac80d51101a44cc91e73835ff0305a1&scope.project.id=968d6c44427c4579bb79f6c800830669&role.id=7cbff51867a7475d837afd51f6c12f11 => generated 610 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:00:00.468506 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=36 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:00.621701 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=80 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:45.502253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f29203-aeb5-4e8b-bad6-a3d362429fd4 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:45.502892 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f29203-aeb5-4e8b-bad6-a3d362429fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:45.502892 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f29203-aeb5-4e8b-bad6-a3d362429fd4 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:45.504497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 605/1201] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:00:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:00:45.510034 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d12013-c33f-426e-ae0a-058da0c1426e None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:45.510503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d12013-c33f-426e-ae0a-058da0c1426e None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:45.510590 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d12013-c33f-426e-ae0a-058da0c1426e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:45.529906 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-d8d12013-c33f-426e-ae0a-058da0c1426e None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:00:45.531258 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d8d12013-c33f-426e-ae0a-058da0c1426e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:45.626008 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 597/1202] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:00:45 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:00:45.650480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f29203-aeb5-4e8b-bad6-a3d362429fd4 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:45.791006 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f29203-aeb5-4e8b-bad6-a3d362429fd4 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:45.840646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e25444-eb94-4378-9893-9fdd05434dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:45.841345 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e25444-eb94-4378-9893-9fdd05434dff None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:45.841634 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e25444-eb94-4378-9893-9fdd05434dff None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:45.842062 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e25444-eb94-4378-9893-9fdd05434dff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:45.859657 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e25444-eb94-4378-9893-9fdd05434dff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:45.861446 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e25444-eb94-4378-9893-9fdd05434dff None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:45.881787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-82e25444-eb94-4378-9893-9fdd05434dff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:00:45.881787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:00:45.881787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:00:45.881787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:00:45.881787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:00:45.881787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:00:45.881787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:00:45.881787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:00:45.881787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:00:45.881787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:00:45.881787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:00:45.881787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:00:45.881787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:00:45.881787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:00:45.881787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:00:45.882860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:00:45.882860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:00:45.882860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:00:45.882860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:00:45.882860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:00:45.882860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:00:45.882860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:00:45.882860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:00:45.882860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 09:00:45.882860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:00:45.882860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:00:45.882860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:00:45.882860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:00:45.882860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:00:45.882860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:00:45.882860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:00:45.882860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:00:45.884286 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:00:45.884286 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:00:45.884286 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:00:45.884286 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 09:00:45.884286 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 606/1203] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 09:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:00:45.890353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d12013-c33f-426e-ae0a-058da0c1426e None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:46.046957 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d12013-c33f-426e-ae0a-058da0c1426e None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:46.082832 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:46.094023 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca47e5-bfc6-45c8-beb4-59a3521d42cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:46.095256 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca47e5-bfc6-45c8-beb4-59a3521d42cd None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:46.095729 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca47e5-bfc6-45c8-beb4-59a3521d42cd None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:46.096074 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca47e5-bfc6-45c8-beb4-59a3521d42cd None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:46.097377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca47e5-bfc6-45c8-beb4-59a3521d42cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:00:46.097704 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca47e5-bfc6-45c8-beb4-59a3521d42cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:46.100333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca47e5-bfc6-45c8-beb4-59a3521d42cd None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:46.116114 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 598/1204] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 09:00:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:00:46.121962 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e25444-eb94-4378-9893-9fdd05434dff None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:46.195099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:46.266150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e25444-eb94-4378-9893-9fdd05434dff None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:46.310455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159191f7-2347-4c90-9005-d6d580203c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:46.311041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159191f7-2347-4c90-9005-d6d580203c5c None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:46.311920 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159191f7-2347-4c90-9005-d6d580203c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:46.312287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159191f7-2347-4c90-9005-d6d580203c5c None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:46.312973 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159191f7-2347-4c90-9005-d6d580203c5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:46.314461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159191f7-2347-4c90-9005-d6d580203c5c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:46.338137 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 607/1205] 10.222.0.100 () {60 vars in 1137 bytes} [Tue Aug 2 09:00:46 2022] POST /identity/v3/users => generated 252 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:00:48.557059 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684994f-c3f2-4213-8544-617ec2525e65 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:48.557731 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684994f-c3f2-4213-8544-617ec2525e65 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:48.557731 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684994f-c3f2-4213-8544-617ec2525e65 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:48.559273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 599/1206] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:00:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:00:48.566515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40a75c-e82f-4139-b33a-d537397bffc5 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:48.566994 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40a75c-e82f-4139-b33a-d537397bffc5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:48.567268 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40a75c-e82f-4139-b33a-d537397bffc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:48.593064 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-fc40a75c-e82f-4139-b33a-d537397bffc5 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:00:48.594312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fc40a75c-e82f-4139-b33a-d537397bffc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:48.681324 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 608/1207] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:00:48 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:00:48.706323 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684994f-c3f2-4213-8544-617ec2525e65 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:48.825548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684994f-c3f2-4213-8544-617ec2525e65 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:48.861880 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b38f2-473e-40ac-8351-b5a3d7feed36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:48.862453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b38f2-473e-40ac-8351-b5a3d7feed36 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:48.862764 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b38f2-473e-40ac-8351-b5a3d7feed36 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:48.862997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b38f2-473e-40ac-8351-b5a3d7feed36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:48.875107 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b38f2-473e-40ac-8351-b5a3d7feed36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:00:48.880769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-178b38f2-473e-40ac-8351-b5a3d7feed36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:00:48.880769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:00:48.880769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:00:48.880769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:00:48.880769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:00:48.880769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:00:48.880769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:00:48.880769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:00:48.880769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:00:48.880769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:00:48.880769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:00:48.880769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:00:48.880769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:00:48.880769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:00:48.880769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:00:48.880769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:00:48.882001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:00:48.882001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:00:48.882001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:00:48.882001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:00:48.882001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:00:48.882001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:00:48.882001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:00:48.882001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:00:48.882001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:00:48.882001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:00:48.882001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:00:48.882001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 09:00:48.882001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:00:48.882001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:00:48.882001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:00:48.882001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:00:48.883284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:00:48.883284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:00:48.883284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:00:48.883284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:00:48.883284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:00:48.883284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:00:48.883284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:00:48.883284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:00:48.883284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:00:48.883284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:00:48.883284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:00:48.883284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:00:48.891851 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40a75c-e82f-4139-b33a-d537397bffc5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:48.897185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 600/1208] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 09:00:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:00:48.939702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:49.016435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40a75c-e82f-4139-b33a-d537397bffc5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:49.055942 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea93c4-6291-4b47-bfa2-368857d5bb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:49.056606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea93c4-6291-4b47-bfa2-368857d5bb20 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:49.056893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea93c4-6291-4b47-bfa2-368857d5bb20 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:49.057194 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea93c4-6291-4b47-bfa2-368857d5bb20 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:49.057910 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea93c4-6291-4b47-bfa2-368857d5bb20 None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:00:49.058477 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea93c4-6291-4b47-bfa2-368857d5bb20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:49.059984 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea93c4-6291-4b47-bfa2-368857d5bb20 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:49.069018 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 609/1209] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 09:00:48 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:00:49.075094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b38f2-473e-40ac-8351-b5a3d7feed36 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:49.135554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:49.195676 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b38f2-473e-40ac-8351-b5a3d7feed36 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:49.240299 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9edef7-dfc2-46be-8bdc-a4a737dc0fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:49.240995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9edef7-dfc2-46be-8bdc-a4a737dc0fef None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:49.241315 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9edef7-dfc2-46be-8bdc-a4a737dc0fef None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:49.241594 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9edef7-dfc2-46be-8bdc-a4a737dc0fef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:49.253442 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9edef7-dfc2-46be-8bdc-a4a737dc0fef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:49.255023 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9edef7-dfc2-46be-8bdc-a4a737dc0fef None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:49.272510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-2f9edef7-dfc2-46be-8bdc-a4a737dc0fef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:00:49.272510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:00:49.272510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:00:49.272510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:00:49.272510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:00:49.272510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:00:49.272510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:00:49.272510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:00:49.272510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:00:49.272510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:00:49.272510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:00:49.272510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:00:49.272510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:00:49.272510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:00:49.272510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:00:49.273706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:00:49.273706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:00:49.273706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:00:49.273706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:00:49.273706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:00:49.273706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:00:49.273706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:00:49.273706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:00:49.273706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 09:00:49.273706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:00:49.273706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:00:49.273706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:00:49.273706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:00:49.273706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:00:49.273706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:00:49.273706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:00:49.273706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:00:49.274641 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:00:49.274641 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:00:49.274641 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:00:49.274641 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:00:49.274641 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 601/1210] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 09:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:00:49.279532 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea93c4-6291-4b47-bfa2-368857d5bb20 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:49.401897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:49.406479 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea93c4-6291-4b47-bfa2-368857d5bb20 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:49.451716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ffa24-6b36-4507-9df4-80ec25b46eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:49.452543 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ffa24-6b36-4507-9df4-80ec25b46eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:49.452830 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ffa24-6b36-4507-9df4-80ec25b46eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:49.453158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ffa24-6b36-4507-9df4-80ec25b46eb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:49.453833 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ffa24-6b36-4507-9df4-80ec25b46eb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:00:49.454175 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ffa24-6b36-4507-9df4-80ec25b46eb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:49.455490 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ffa24-6b36-4507-9df4-80ec25b46eb0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:49.469573 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 610/1211] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 09:00:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:00:49.475109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9edef7-dfc2-46be-8bdc-a4a737dc0fef None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:49.588257 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9edef7-dfc2-46be-8bdc-a4a737dc0fef None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:49.590091 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:49.628394 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db3c68-03f5-489a-9179-cdf29a865851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:49.629059 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db3c68-03f5-489a-9179-cdf29a865851 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:49.629381 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db3c68-03f5-489a-9179-cdf29a865851 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:49.629769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db3c68-03f5-489a-9179-cdf29a865851 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:49.640491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db3c68-03f5-489a-9179-cdf29a865851 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:49.642097 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db3c68-03f5-489a-9179-cdf29a865851 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:49.659866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-63db3c68-03f5-489a-9179-cdf29a865851 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:00:49.659866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:00:49.659866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:00:49.659866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:00:49.659866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:00:49.659866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:00:49.659866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:00:49.659866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:00:49.659866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:00:49.659866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:00:49.659866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:00:49.659866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:00:49.659866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:00:49.659866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:00:49.659866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:00:49.659866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:00:49.661089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:00:49.661089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:00:49.661089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:00:49.661089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:00:49.661089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:00:49.661089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:00:49.661089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:00:49.661089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:00:49.661089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:00:49.661089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:00:49.661089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:00:49.661089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 09:00:49.661089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:00:49.661089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:00:49.661089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:00:49.661089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:00:49.661089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:00:49.661089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:00:49.662416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:00:49.662416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:00:49.662416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:00:49.662416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:00:49.662416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:00:49.662416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:00:49.662416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:00:49.662416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:00:49.662416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:00:49.662416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:00:49.662416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 602/1212] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 09:00:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:00:49.667146 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ffa24-6b36-4507-9df4-80ec25b46eb0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:49.819300 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ffa24-6b36-4507-9df4-80ec25b46eb0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:49.870867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ee22c-ac06-4734-a2a2-94b25629c6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:49.872841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ee22c-ac06-4734-a2a2-94b25629c6b6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:49.872841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ee22c-ac06-4734-a2a2-94b25629c6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:49.872841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ee22c-ac06-4734-a2a2-94b25629c6b6 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:49.873585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ee22c-ac06-4734-a2a2-94b25629c6b6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:00:49.873958 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ee22c-ac06-4734-a2a2-94b25629c6b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:49.875574 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ee22c-ac06-4734-a2a2-94b25629c6b6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:49.901915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 611/1213] 10.222.0.100 () {56 vars in 1148 bytes} [Tue Aug 2 09:00:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:00:49.906147 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db3c68-03f5-489a-9179-cdf29a865851 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:50.049043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db3c68-03f5-489a-9179-cdf29a865851 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:50.097798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e108a-b5a9-473f-8639-320f13fbe0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:50.098455 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e108a-b5a9-473f-8639-320f13fbe0bc None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:50.098791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e108a-b5a9-473f-8639-320f13fbe0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:50.098999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e108a-b5a9-473f-8639-320f13fbe0bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:50.134746 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:50.183231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e108a-b5a9-473f-8639-320f13fbe0bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:50.184689 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e108a-b5a9-473f-8639-320f13fbe0bc None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:50.300062 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 603/1214] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 09:00:49 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:00:50.305867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ee22c-ac06-4734-a2a2-94b25629c6b6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:50.326015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:50.416399 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ee22c-ac06-4734-a2a2-94b25629c6b6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:50.457718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ae6c7-17a7-44f8-84a6-141c936e62f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:50.458283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ae6c7-17a7-44f8-84a6-141c936e62f2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:50.458564 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ae6c7-17a7-44f8-84a6-141c936e62f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:50.458841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ae6c7-17a7-44f8-84a6-141c936e62f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:50.470311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ae6c7-17a7-44f8-84a6-141c936e62f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:50.471892 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ae6c7-17a7-44f8-84a6-141c936e62f2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:50.489712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-8c2ae6c7-17a7-44f8-84a6-141c936e62f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:00:50.489712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:00:50.489712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:00:50.489712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:00:50.489712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:00:50.489712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:00:50.489712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:00:50.489712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:00:50.489712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:00:50.489712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:00:50.489712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:00:50.489712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:00:50.489712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:00:50.489712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:00:50.489712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:00:50.490931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:00:50.490931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:00:50.490931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:00:50.490931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:00:50.490931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:00:50.490931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:00:50.490931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:00:50.490931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:00:50.490931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 09:00:50.490931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:00:50.490931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:00:50.490931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:00:50.490931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:00:50.490931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:00:50.490931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:00:50.490931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:00:50.490931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:00:50.492285 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:00:50.492285 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:00:50.492285 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:00:50.492285 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 09:00:50.495657 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 612/1215] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 09:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:00:50.497063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e108a-b5a9-473f-8639-320f13fbe0bc None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:50.567813 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:50.616625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e108a-b5a9-473f-8639-320f13fbe0bc None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:50.658376 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404f4c0-76b7-44c6-9e3a-b83df181fcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:50.658988 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404f4c0-76b7-44c6-9e3a-b83df181fcd5 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:50.659269 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404f4c0-76b7-44c6-9e3a-b83df181fcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:50.659530 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404f4c0-76b7-44c6-9e3a-b83df181fcd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:50.660218 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404f4c0-76b7-44c6-9e3a-b83df181fcd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:00:50.660552 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404f4c0-76b7-44c6-9e3a-b83df181fcd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:50.661921 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404f4c0-76b7-44c6-9e3a-b83df181fcd5 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:50.674743 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 604/1216] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 09:00:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:00:50.680771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ae6c7-17a7-44f8-84a6-141c936e62f2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:50.762947 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:50.798783 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ae6c7-17a7-44f8-84a6-141c936e62f2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:50.837752 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8ad26-42f5-411d-86fd-05dbbcbaf0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:50.838465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8ad26-42f5-411d-86fd-05dbbcbaf0db None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:50.838704 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8ad26-42f5-411d-86fd-05dbbcbaf0db None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:50.838992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8ad26-42f5-411d-86fd-05dbbcbaf0db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:50.847911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8ad26-42f5-411d-86fd-05dbbcbaf0db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:50.849585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8ad26-42f5-411d-86fd-05dbbcbaf0db None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:50.864216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-18d8ad26-42f5-411d-86fd-05dbbcbaf0db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:00:50.864216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:00:50.864216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:00:50.864216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:00:50.864216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:00:50.864216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:00:50.864216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:00:50.864216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:00:50.864216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:00:50.864216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:00:50.864216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:00:50.864216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:00:50.864216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:00:50.864216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:00:50.864216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:00:50.865702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:00:50.865702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:00:50.865702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:00:50.865702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:00:50.865702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:00:50.865702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:00:50.865702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:00:50.865702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:00:50.865702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 09:00:50.865702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:00:50.865702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:00:50.865702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:00:50.865702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:00:50.865702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:00:50.865702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:00:50.865702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:00:50.865702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:00:50.866840 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:00:50.866840 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:00:50.866840 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:00:50.866840 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:00:50.866840 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:00:50.866840 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:00:50.866840 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:00:50.866840 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 09:00:50.871635 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404f4c0-76b7-44c6-9e3a-b83df181fcd5 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:50.875206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 613/1217] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 09:00:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:00:50.986750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404f4c0-76b7-44c6-9e3a-b83df181fcd5 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:51.016978 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:51.032612 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583e661-cd0b-4d3e-8776-98c75d1fb36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:51.033388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583e661-cd0b-4d3e-8776-98c75d1fb36c None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:51.033757 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583e661-cd0b-4d3e-8776-98c75d1fb36c None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:51.034159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583e661-cd0b-4d3e-8776-98c75d1fb36c None admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:51.035013 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583e661-cd0b-4d3e-8776-98c75d1fb36c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:00:51.035392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583e661-cd0b-4d3e-8776-98c75d1fb36c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:51.037248 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583e661-cd0b-4d3e-8776-98c75d1fb36c None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:51.052444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 605/1218] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 09:00:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:00:51.059189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8ad26-42f5-411d-86fd-05dbbcbaf0db None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:51.173382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8ad26-42f5-411d-86fd-05dbbcbaf0db None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:51.203841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:51.214737 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6c04b-c5f1-47e3-bf93-de5a66c258f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:51.214917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6c04b-c5f1-47e3-bf93-de5a66c258f3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:51.215111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6c04b-c5f1-47e3-bf93-de5a66c258f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:51.215523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6c04b-c5f1-47e3-bf93-de5a66c258f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:51.216324 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6c04b-c5f1-47e3-bf93-de5a66c258f3 None admin] RBAC: Adding query filter params (role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=b40981fe179a43dc81553d3594dad649) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:00:51.216632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6c04b-c5f1-47e3-bf93-de5a66c258f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:51.218271 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6c04b-c5f1-47e3-bf93-de5a66c258f3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:51.224239 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 614/1219] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 09:00:51 2022] GET /identity/v3/role_assignments?user.id=b40981fe179a43dc81553d3594dad649&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:00:53.228271 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af0d0c-3796-4f12-aba9-c653d94080fe None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:53.228843 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af0d0c-3796-4f12-aba9-c653d94080fe None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:53.228843 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af0d0c-3796-4f12-aba9-c653d94080fe None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:53.230289 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 606/1220] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:00:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 09:00:53.235948 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394031cc-662a-471b-8416-865e3ad40437 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:53.236407 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394031cc-662a-471b-8416-865e3ad40437 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:53.236544 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394031cc-662a-471b-8416-865e3ad40437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:53.255942 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-394031cc-662a-471b-8416-865e3ad40437 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:00:53.257309 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-394031cc-662a-471b-8416-865e3ad40437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:53.327947 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 615/1221] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:00:53 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:00:53.351336 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af0d0c-3796-4f12-aba9-c653d94080fe None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:53.470516 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af0d0c-3796-4f12-aba9-c653d94080fe None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:53.509437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc3e67-16d7-4389-9da9-07eebf805948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:53.510069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc3e67-16d7-4389-9da9-07eebf805948 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:53.510346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc3e67-16d7-4389-9da9-07eebf805948 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:53.510616 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc3e67-16d7-4389-9da9-07eebf805948 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:53.522010 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc3e67-16d7-4389-9da9-07eebf805948 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:00:53.526936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-17bc3e67-16d7-4389-9da9-07eebf805948 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:00:53.526936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:00:53.526936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:00:53.526936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:00:53.526936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:00:53.526936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:00:53.526936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:00:53.526936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:00:53.526936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:00:53.526936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:00:53.526936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:00:53.526936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:00:53.526936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:00:53.526936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:00:53.526936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:00:53.526936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:00:53.528056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:00:53.528056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:00:53.528056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:00:53.528056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:00:53.528056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:00:53.528056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:00:53.528056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:00:53.528056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:00:53.528056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:00:53.528056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:00:53.528056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:00:53.528056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 09:00:53.528056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:00:53.528056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:00:53.528056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:00:53.528056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:00:53.529090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:00:53.529090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:00:53.529090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:00:53.529090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:00:53.529090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:00:53.529090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:00:53.529090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:00:53.529090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:00:53.529090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:00:53.529090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:00:53.529090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:00:53.529090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:00:53.529090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 607/1222] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 09:00:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:00:53.535219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394031cc-662a-471b-8416-865e3ad40437 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:53.658734 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394031cc-662a-471b-8416-865e3ad40437 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:53.687898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:53.703240 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4ad7f-6dd8-4d26-b3f0-f687a0f6a57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:53.703974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4ad7f-6dd8-4d26-b3f0-f687a0f6a57c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:53.704287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4ad7f-6dd8-4d26-b3f0-f687a0f6a57c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:53.704618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4ad7f-6dd8-4d26-b3f0-f687a0f6a57c None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:53.706076 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4ad7f-6dd8-4d26-b3f0-f687a0f6a57c None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:00:53.706389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4ad7f-6dd8-4d26-b3f0-f687a0f6a57c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:53.707984 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4ad7f-6dd8-4d26-b3f0-f687a0f6a57c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:53.717624 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 616/1223] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 09:00:53 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:00:53.723568 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc3e67-16d7-4389-9da9-07eebf805948 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:53.774584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:53.839450 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc3e67-16d7-4389-9da9-07eebf805948 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:53.898594 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fefcb-f91a-4ee0-b440-70a2442ed602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:53.899638 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fefcb-f91a-4ee0-b440-70a2442ed602 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:53.900119 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fefcb-f91a-4ee0-b440-70a2442ed602 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:53.900597 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fefcb-f91a-4ee0-b440-70a2442ed602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:53.913972 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fefcb-f91a-4ee0-b440-70a2442ed602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:53.915673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fefcb-f91a-4ee0-b440-70a2442ed602 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:53.933983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-a08fefcb-f91a-4ee0-b440-70a2442ed602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:00:53.933983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:00:53.933983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:00:53.933983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:00:53.933983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:00:53.933983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:00:53.933983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:00:53.933983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:00:53.933983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:00:53.933983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:00:53.933983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:00:53.933983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:00:53.933983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:00:53.933983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:00:53.933983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:00:53.935052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:00:53.935052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:00:53.935052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:00:53.935052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:00:53.935052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:00:53.935052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:00:53.935052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:00:53.935052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:00:53.935052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 09:00:53.935052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:00:53.935052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:00:53.935052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:00:53.935052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:00:53.935052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:00:53.935052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:00:53.935052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:00:53.935052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:00:53.936625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:00:53.936625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:00:53.936625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:00:53.936625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:00:53.936625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 608/1224] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 09:00:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:00:53.942126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4ad7f-6dd8-4d26-b3f0-f687a0f6a57c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:54.049554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4ad7f-6dd8-4d26-b3f0-f687a0f6a57c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:54.088028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfacbf3-9285-4645-8ed5-eed59d62bc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:54.088673 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfacbf3-9285-4645-8ed5-eed59d62bc49 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:54.088956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfacbf3-9285-4645-8ed5-eed59d62bc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:54.089267 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfacbf3-9285-4645-8ed5-eed59d62bc49 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:54.089952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfacbf3-9285-4645-8ed5-eed59d62bc49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:00:54.090258 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfacbf3-9285-4645-8ed5-eed59d62bc49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:54.091612 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfacbf3-9285-4645-8ed5-eed59d62bc49 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:54.104725 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 617/1225] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 09:00:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:00:54.110954 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fefcb-f91a-4ee0-b440-70a2442ed602 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:54.215109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:54.222885 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fefcb-f91a-4ee0-b440-70a2442ed602 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:54.259799 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e92c2-e6a0-42d2-a1e0-88865aaff649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:54.260443 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e92c2-e6a0-42d2-a1e0-88865aaff649 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:54.260709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e92c2-e6a0-42d2-a1e0-88865aaff649 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:54.260986 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e92c2-e6a0-42d2-a1e0-88865aaff649 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:54.268711 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982e92c2-e6a0-42d2-a1e0-88865aaff649 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:54.270190 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982e92c2-e6a0-42d2-a1e0-88865aaff649 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:54.282820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-982e92c2-e6a0-42d2-a1e0-88865aaff649 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:00:54.282820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:00:54.282820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:00:54.282820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:00:54.282820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:00:54.282820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:00:54.282820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:00:54.282820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:00:54.282820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:00:54.282820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:00:54.282820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:00:54.282820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:00:54.282820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:00:54.282820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:00:54.282820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:00:54.282820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:00:54.284047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:00:54.284047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:00:54.284047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:00:54.284047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:00:54.284047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:00:54.284047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:00:54.284047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:00:54.284047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:00:54.284047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:00:54.284047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:00:54.284047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:00:54.284047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 09:00:54.284047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:00:54.284047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:00:54.284047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:00:54.284047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:00:54.284047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:00:54.284047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:00:54.285030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:00:54.285030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:00:54.285030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:00:54.285030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:00:54.285030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:00:54.285030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:00:54.285030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:00:54.285030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:00:54.285030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:00:54.285030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:00:54.285030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 609/1226] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 09:00:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:00:54.290425 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfacbf3-9285-4645-8ed5-eed59d62bc49 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:54.346861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:54.404141 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfacbf3-9285-4645-8ed5-eed59d62bc49 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:54.440647 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bb1fd-d83d-49c7-9441-a2d4e07b997b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:54.441445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bb1fd-d83d-49c7-9441-a2d4e07b997b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:54.441860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bb1fd-d83d-49c7-9441-a2d4e07b997b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:54.442248 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bb1fd-d83d-49c7-9441-a2d4e07b997b None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:54.443218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bb1fd-d83d-49c7-9441-a2d4e07b997b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:00:54.443628 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bb1fd-d83d-49c7-9441-a2d4e07b997b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:54.445144 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bb1fd-d83d-49c7-9441-a2d4e07b997b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:54.462594 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 618/1227] 10.222.0.100 () {56 vars in 1148 bytes} [Tue Aug 2 09:00:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:00:54.469341 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e92c2-e6a0-42d2-a1e0-88865aaff649 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:54.580185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e92c2-e6a0-42d2-a1e0-88865aaff649 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:54.621134 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d7f10-6755-40df-8c39-e85201958ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:54.621718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d7f10-6755-40df-8c39-e85201958ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:54.622001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d7f10-6755-40df-8c39-e85201958ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:54.622277 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d7f10-6755-40df-8c39-e85201958ce8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:54.633395 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d7f10-6755-40df-8c39-e85201958ce8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:54.634843 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d7f10-6755-40df-8c39-e85201958ce8 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:54.649480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-185d7f10-6755-40df-8c39-e85201958ce8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:00:54.649480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:00:54.649480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:00:54.649480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:00:54.649480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:00:54.649480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:00:54.649480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:00:54.649480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:00:54.649480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:00:54.649480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:00:54.649480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:00:54.649480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:00:54.649480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:00:54.649480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:00:54.649480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:00:54.650518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:00:54.650518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:00:54.650518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:00:54.650518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:00:54.650518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:00:54.650518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:00:54.650518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:00:54.650518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:00:54.650518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 09:00:54.650518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:00:54.650518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:00:54.650518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:00:54.650518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:00:54.650518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:00:54.650518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:00:54.650518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:00:54.650518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:00:54.651441 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:00:54.651441 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:00:54.651441 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:00:54.651441 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:00:54.651441 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 610/1228] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 09:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:00:54.657378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bb1fd-d83d-49c7-9441-a2d4e07b997b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:54.765614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bb1fd-d83d-49c7-9441-a2d4e07b997b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:54.768412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:54.809718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f314a-583c-4caa-bb92-789d40481033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:54.810338 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f314a-583c-4caa-bb92-789d40481033 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:54.811268 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f314a-583c-4caa-bb92-789d40481033 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:54.811572 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f314a-583c-4caa-bb92-789d40481033 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:54.812348 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f314a-583c-4caa-bb92-789d40481033 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:00:54.812722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f314a-583c-4caa-bb92-789d40481033 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:54.814259 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f314a-583c-4caa-bb92-789d40481033 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:54.828311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 619/1229] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 09:00:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:00:54.837021 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d7f10-6755-40df-8c39-e85201958ce8 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:54.846511 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:54.956868 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d7f10-6755-40df-8c39-e85201958ce8 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:54.993333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadecc5-e51d-4d55-9057-2f1e6ce02d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:54.993974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadecc5-e51d-4d55-9057-2f1e6ce02d8b None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:54.994324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadecc5-e51d-4d55-9057-2f1e6ce02d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:54.994660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadecc5-e51d-4d55-9057-2f1e6ce02d8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:55.002552 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fadecc5-e51d-4d55-9057-2f1e6ce02d8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:55.004109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fadecc5-e51d-4d55-9057-2f1e6ce02d8b None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:55.016953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-1fadecc5-e51d-4d55-9057-2f1e6ce02d8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:00:55.016953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:00:55.016953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:00:55.016953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:00:55.016953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:00:55.016953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:00:55.016953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:00:55.016953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:00:55.016953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:00:55.016953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:00:55.016953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:00:55.016953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:00:55.016953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:00:55.016953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:00:55.016953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:00:55.018316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:00:55.018316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:00:55.018316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:00:55.018316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:00:55.018316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:00:55.018316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:00:55.018316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:00:55.018316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:00:55.018316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 09:00:55.018316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:00:55.018316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:00:55.018316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:00:55.018316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:00:55.018316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:00:55.018316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:00:55.018316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:00:55.018316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:00:55.019721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:00:55.019721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:00:55.019721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:00:55.019721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:00:55.019721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:00:55.019721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:00:55.019721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:00:55.019721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:00:55.019721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 611/1230] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 09:00:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:00:55.024881 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f314a-583c-4caa-bb92-789d40481033 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:55.140958 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f314a-583c-4caa-bb92-789d40481033 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:55.178892 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015929d-5f51-4070-8088-c2e11eb6f348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:55.179521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015929d-5f51-4070-8088-c2e11eb6f348 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:55.179799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015929d-5f51-4070-8088-c2e11eb6f348 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:55.180077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015929d-5f51-4070-8088-c2e11eb6f348 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:55.180817 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015929d-5f51-4070-8088-c2e11eb6f348 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:00:55.181153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015929d-5f51-4070-8088-c2e11eb6f348 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:55.182566 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015929d-5f51-4070-8088-c2e11eb6f348 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:55.195495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 620/1231] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 09:00:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:00:55.202359 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadecc5-e51d-4d55-9057-2f1e6ce02d8b None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:55.278005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:55.317567 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadecc5-e51d-4d55-9057-2f1e6ce02d8b None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:55.359785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd112ba-d1b6-448d-9cab-cfbec7821c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:55.360455 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd112ba-d1b6-448d-9cab-cfbec7821c5e None admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:55.360787 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd112ba-d1b6-448d-9cab-cfbec7821c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:55.361009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd112ba-d1b6-448d-9cab-cfbec7821c5e None admin] PATH_INFO: `/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/b40981fe179a43dc81553d3594dad649/roles/6e6ac8e8e7044b209d1a8b5bb6e4b1d8` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:55.376489 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd112ba-d1b6-448d-9cab-cfbec7821c5e None admin] RBAC: Authorizing `identity:create_grant(project_id=61ca0176324e445b94e42a5214d2c89b, user_id=b40981fe179a43dc81553d3594dad649, role_id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:55.377964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd112ba-d1b6-448d-9cab-cfbec7821c5e None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:55.390847 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 612/1232] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 09:00:55 2022] PUT /identity/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/b40981fe179a43dc81553d3594dad649/roles/6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:00:57.569200 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2bdf3-20cf-43f6-b098-6f63cf95b6f0 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:57.569949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2bdf3-20cf-43f6-b098-6f63cf95b6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:57.569949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2bdf3-20cf-43f6-b098-6f63cf95b6f0 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:57.571299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 621/1233] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:00:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:00:57.577649 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db0d83-c23c-46c1-8ba0-ce6f93b2d6f6 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:57.578143 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db0d83-c23c-46c1-8ba0-ce6f93b2d6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:57.578258 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db0d83-c23c-46c1-8ba0-ce6f93b2d6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:57.600539 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-00db0d83-c23c-46c1-8ba0-ce6f93b2d6f6 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:00:57.602144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-00db0d83-c23c-46c1-8ba0-ce6f93b2d6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:57.675014 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 613/1234] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:00:57 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:00:57.698123 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2bdf3-20cf-43f6-b098-6f63cf95b6f0 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:57.725094 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:57.810209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2bdf3-20cf-43f6-b098-6f63cf95b6f0 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:57.847487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf36d39-c4d5-4c1b-8948-a17238eb8a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:57.847487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf36d39-c4d5-4c1b-8948-a17238eb8a93 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:57.847766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf36d39-c4d5-4c1b-8948-a17238eb8a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:57.847961 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf36d39-c4d5-4c1b-8948-a17238eb8a93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:57.860938 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf36d39-c4d5-4c1b-8948-a17238eb8a93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:00:57.866733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-edf36d39-c4d5-4c1b-8948-a17238eb8a93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:00:57.866733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:00:57.866733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:00:57.866733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:00:57.866733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:00:57.866733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:00:57.866733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:00:57.866733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:00:57.866733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:00:57.866733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:00:57.866733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:00:57.866733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:00:57.866733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:00:57.866733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:00:57.866733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:00:57.866733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:00:57.867854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:00:57.867854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:00:57.867854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:00:57.867854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:00:57.867854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:00:57.867854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:00:57.867854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:00:57.867854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:00:57.867854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:00:57.867854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:00:57.867854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:00:57.867854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 09:00:57.867854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:00:57.867854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:00:57.867854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:00:57.867854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:00:57.868956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:00:57.868956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:00:57.868956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:00:57.868956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:00:57.868956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:00:57.868956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:00:57.868956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:00:57.868956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:00:57.868956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:00:57.868956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:00:57.868956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:00:57.868956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 09:00:57.868956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 622/1235] 10.222.0.100 () {56 vars in 1107 bytes} [Tue Aug 2 09:00:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:00:57.875602 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db0d83-c23c-46c1-8ba0-ce6f93b2d6f6 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:57.923356 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:57.981189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db0d83-c23c-46c1-8ba0-ce6f93b2d6f6 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:58.018821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d170a-d519-4de0-a258-b76864ebdc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:58.019495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d170a-d519-4de0-a258-b76864ebdc68 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:58.019718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d170a-d519-4de0-a258-b76864ebdc68 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:58.019990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d170a-d519-4de0-a258-b76864ebdc68 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:58.020699 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d170a-d519-4de0-a258-b76864ebdc68 None admin] RBAC: Adding query filter params (name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:00:58.021004 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d170a-d519-4de0-a258-b76864ebdc68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:58.022414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d170a-d519-4de0-a258-b76864ebdc68 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:58.031642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 614/1236] 10.222.0.100 () {56 vars in 1108 bytes} [Tue Aug 2 09:00:57 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:00:58.037341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf36d39-c4d5-4c1b-8948-a17238eb8a93 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:58.127505 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:58.145208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf36d39-c4d5-4c1b-8948-a17238eb8a93 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:58.184204 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc10851-73c5-4c66-9b08-01dedc55f4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:58.184900 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc10851-73c5-4c66-9b08-01dedc55f4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:58.185258 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc10851-73c5-4c66-9b08-01dedc55f4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:58.185673 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc10851-73c5-4c66-9b08-01dedc55f4c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:58.196895 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc10851-73c5-4c66-9b08-01dedc55f4c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:58.198370 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc10851-73c5-4c66-9b08-01dedc55f4c4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:58.214140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-2dc10851-73c5-4c66-9b08-01dedc55f4c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:00:58.214140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:00:58.214140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:00:58.214140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:00:58.214140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:00:58.214140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:00:58.214140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:00:58.214140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:00:58.214140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:00:58.214140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:00:58.214140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:00:58.214140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:00:58.214140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:00:58.214140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:00:58.214140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:00:58.215283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:00:58.215283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:00:58.215283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:00:58.215283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:00:58.215283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:00:58.215283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:00:58.215283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:00:58.215283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:00:58.215283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 09:00:58.215283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:00:58.215283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:00:58.215283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:00:58.215283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:00:58.215283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:00:58.215283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:00:58.215283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:00:58.215283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:00:58.216408 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:00:58.216408 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:00:58.216408 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:00:58.216408 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 09:00:58.216408 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 623/1237] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 09:00:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:00:58.222083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d170a-d519-4de0-a258-b76864ebdc68 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:58.338771 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d170a-d519-4de0-a258-b76864ebdc68 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:58.348814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:58.378688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26dc41-63ed-4f94-b680-13902cc4b219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:58.379244 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26dc41-63ed-4f94-b680-13902cc4b219 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:58.379529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26dc41-63ed-4f94-b680-13902cc4b219 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:58.379808 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26dc41-63ed-4f94-b680-13902cc4b219 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:58.380499 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26dc41-63ed-4f94-b680-13902cc4b219 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:00:58.380807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26dc41-63ed-4f94-b680-13902cc4b219 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:58.382329 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26dc41-63ed-4f94-b680-13902cc4b219 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:58.396623 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 615/1238] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 09:00:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:00:58.402530 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc10851-73c5-4c66-9b08-01dedc55f4c4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:58.522685 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc10851-73c5-4c66-9b08-01dedc55f4c4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:58.560514 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3fdc9-4019-46e2-85f1-31274f7aad71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:58.561268 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3fdc9-4019-46e2-85f1-31274f7aad71 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:58.561556 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3fdc9-4019-46e2-85f1-31274f7aad71 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:58.561836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3fdc9-4019-46e2-85f1-31274f7aad71 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:58.565004 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:58.572076 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3fdc9-4019-46e2-85f1-31274f7aad71 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:58.574169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3fdc9-4019-46e2-85f1-31274f7aad71 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:58.586988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-d1b3fdc9-4019-46e2-85f1-31274f7aad71 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:00:58.586988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:00:58.586988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:00:58.586988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:00:58.586988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:00:58.586988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:00:58.586988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:00:58.586988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:00:58.586988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:00:58.586988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:00:58.586988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:00:58.586988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:00:58.586988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:00:58.586988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:00:58.586988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:00:58.586988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:00:58.588116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:00:58.588116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:00:58.588116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:00:58.588116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:00:58.588116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:00:58.588116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:00:58.588116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:00:58.588116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:00:58.588116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:00:58.588116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:00:58.588116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:00:58.588116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 09:00:58.588116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:00:58.588116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:00:58.588116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:00:58.588116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:00:58.588116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:00:58.588116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:00:58.589488 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:00:58.589488 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:00:58.589488 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:00:58.589488 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:00:58.589488 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:00:58.589488 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:00:58.589488 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:00:58.589488 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:00:58.589488 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:00:58.589488 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 09:00:58.589488 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 624/1239] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 09:00:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:00:58.603009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26dc41-63ed-4f94-b680-13902cc4b219 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:58.715051 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26dc41-63ed-4f94-b680-13902cc4b219 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:58.754090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc837e-b081-4134-a99d-87a7712fd187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:58.754710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc837e-b081-4134-a99d-87a7712fd187 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:58.754992 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc837e-b081-4134-a99d-87a7712fd187 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:58.755267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc837e-b081-4134-a99d-87a7712fd187 None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:58.756054 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc837e-b081-4134-a99d-87a7712fd187 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:00:58.756359 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc837e-b081-4134-a99d-87a7712fd187 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:58.757953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc837e-b081-4134-a99d-87a7712fd187 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:58.782753 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 616/1240] 10.222.0.100 () {56 vars in 1148 bytes} [Tue Aug 2 09:00:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:00:58.788445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3fdc9-4019-46e2-85f1-31274f7aad71 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:58.899834 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3fdc9-4019-46e2-85f1-31274f7aad71 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:58.940250 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae93ad-fad3-4ca7-9bda-c9fe98023b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:58.940908 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae93ad-fad3-4ca7-9bda-c9fe98023b2c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:58.941231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae93ad-fad3-4ca7-9bda-c9fe98023b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:58.941510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae93ad-fad3-4ca7-9bda-c9fe98023b2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:59.013384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:59.023246 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae93ad-fad3-4ca7-9bda-c9fe98023b2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:59.024904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae93ad-fad3-4ca7-9bda-c9fe98023b2c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:59.140027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 625/1241] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 09:00:58 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:00:59.143703 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc837e-b081-4134-a99d-87a7712fd187 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:59.174093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:59.253057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc837e-b081-4134-a99d-87a7712fd187 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:59.292878 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796a78b-ee53-48f9-83ab-ce71aca11a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:59.293481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796a78b-ee53-48f9-83ab-ce71aca11a09 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:59.293761 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796a78b-ee53-48f9-83ab-ce71aca11a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:59.294035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796a78b-ee53-48f9-83ab-ce71aca11a09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:59.305705 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796a78b-ee53-48f9-83ab-ce71aca11a09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:59.307553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796a78b-ee53-48f9-83ab-ce71aca11a09 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:59.324886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-f796a78b-ee53-48f9-83ab-ce71aca11a09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:00:59.324886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:00:59.324886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:00:59.324886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:00:59.324886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:00:59.324886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:00:59.324886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:00:59.324886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:00:59.324886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:00:59.324886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:00:59.324886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:00:59.324886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:00:59.324886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:00:59.324886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:00:59.324886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:00:59.326081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:00:59.326081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:00:59.326081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:00:59.326081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:00:59.326081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:00:59.326081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:00:59.326081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:00:59.326081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:00:59.326081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 09:00:59.326081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:00:59.326081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:00:59.326081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:00:59.326081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:00:59.326081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:00:59.326081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:00:59.326081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:00:59.326081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:00:59.327094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:00:59.327094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:00:59.327094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:00:59.327094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:00:59.327094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 617/1242] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 09:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:00:59.332218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae93ad-fad3-4ca7-9bda-c9fe98023b2c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:59.455037 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:59.461711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae93ad-fad3-4ca7-9bda-c9fe98023b2c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:59.500220 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b062bc0-46be-4e17-94a2-b544dfb402fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:59.500854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b062bc0-46be-4e17-94a2-b544dfb402fa None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:59.501165 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b062bc0-46be-4e17-94a2-b544dfb402fa None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:59.501451 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b062bc0-46be-4e17-94a2-b544dfb402fa None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:59.502145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b062bc0-46be-4e17-94a2-b544dfb402fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:00:59.502454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b062bc0-46be-4e17-94a2-b544dfb402fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:59.503810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b062bc0-46be-4e17-94a2-b544dfb402fa None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:59.515261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 626/1243] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 09:00:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:00:59.521840 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796a78b-ee53-48f9-83ab-ce71aca11a09 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:59.614498 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:59.640775 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796a78b-ee53-48f9-83ab-ce71aca11a09 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:59.680207 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60155d77-a754-4ed3-89a1-c2300d732e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:59.680848 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60155d77-a754-4ed3-89a1-c2300d732e28 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:59.681137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60155d77-a754-4ed3-89a1-c2300d732e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:59.681418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60155d77-a754-4ed3-89a1-c2300d732e28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:59.688834 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60155d77-a754-4ed3-89a1-c2300d732e28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:59.690436 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60155d77-a754-4ed3-89a1-c2300d732e28 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:59.702338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-60155d77-a754-4ed3-89a1-c2300d732e28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:00:59.702338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:00:59.702338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:00:59.702338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:00:59.702338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:00:59.702338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:00:59.702338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:00:59.702338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:00:59.702338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:00:59.702338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:00:59.702338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:00:59.702338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:00:59.702338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:00:59.702338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:00:59.702338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:00:59.703428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:00:59.703428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:00:59.703428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:00:59.703428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:00:59.703428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:00:59.703428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:00:59.703428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:00:59.703428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:00:59.703428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 09:00:59.703428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:00:59.703428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:00:59.703428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:00:59.703428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:00:59.703428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:00:59.703428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:00:59.703428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:00:59.703428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:00:59.704558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:00:59.704558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:00:59.704558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:00:59.704558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:00:59.704558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:00:59.704558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:00:59.704558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:00:59.704558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:00:59.704558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 618/1244] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 09:00:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:00:59.709940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b062bc0-46be-4e17-94a2-b544dfb402fa None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:59.821998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b062bc0-46be-4e17-94a2-b544dfb402fa None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:59.859839 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4f0a6-b26c-436f-9b8f-d4f7aa3e03f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:59.860460 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4f0a6-b26c-436f-9b8f-d4f7aa3e03f5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:59.860734 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4f0a6-b26c-436f-9b8f-d4f7aa3e03f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:59.861018 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4f0a6-b26c-436f-9b8f-d4f7aa3e03f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:59.861771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4f0a6-b26c-436f-9b8f-d4f7aa3e03f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:00:59.862124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4f0a6-b26c-436f-9b8f-d4f7aa3e03f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:59.863727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4f0a6-b26c-436f-9b8f-d4f7aa3e03f5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:59.876582 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 627/1245] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 09:00:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:00:59.883687 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60155d77-a754-4ed3-89a1-c2300d732e28 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:59.996419 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60155d77-a754-4ed3-89a1-c2300d732e28 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:00.034403 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f15a04-d292-48d7-8bc6-c7119be000ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:00.035161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f15a04-d292-48d7-8bc6-c7119be000ae None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:00.035480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f15a04-d292-48d7-8bc6-c7119be000ae None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:00.035865 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f15a04-d292-48d7-8bc6-c7119be000ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:00.036796 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f15a04-d292-48d7-8bc6-c7119be000ae None admin] RBAC: Adding query filter params (role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=b40981fe179a43dc81553d3594dad649) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:00.037130 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f15a04-d292-48d7-8bc6-c7119be000ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:00.038573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f15a04-d292-48d7-8bc6-c7119be000ae None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:00.045429 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 619/1246] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 09:00:59 2022] GET /identity/v3/role_assignments?user.id=b40981fe179a43dc81553d3594dad649&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=6e6ac8e8e7044b209d1a8b5bb6e4b1d8 => generated 610 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:01:02.119453 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04cccf-3c09-4d2a-9d31-4126b640d007 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:02.119987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04cccf-3c09-4d2a-9d31-4126b640d007 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:02.120102 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04cccf-3c09-4d2a-9d31-4126b640d007 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:02.121767 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 628/1247] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:01:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:01:02.127062 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624500c-dc6c-45a7-a307-da4e22f14666 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:02.127605 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624500c-dc6c-45a7-a307-da4e22f14666 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:02.127743 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624500c-dc6c-45a7-a307-da4e22f14666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:02.134495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:02.148717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-b624500c-dc6c-45a7-a307-da4e22f14666 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:01:02.150152 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b624500c-dc6c-45a7-a307-da4e22f14666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:01:02.230071 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 620/1248] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:01:02 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:01:02.255270 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04cccf-3c09-4d2a-9d31-4126b640d007 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:02.280222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:02.375043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04cccf-3c09-4d2a-9d31-4126b640d007 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:02.414374 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c132d5-5f9c-4fa5-8d19-ebc03edc6d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:02.415026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c132d5-5f9c-4fa5-8d19-ebc03edc6d1b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:02.415311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c132d5-5f9c-4fa5-8d19-ebc03edc6d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:02.415588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c132d5-5f9c-4fa5-8d19-ebc03edc6d1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:02.426397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c132d5-5f9c-4fa5-8d19-ebc03edc6d1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:01:02.431334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-08c132d5-5f9c-4fa5-8d19-ebc03edc6d1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:01:02.431334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:02.431334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:01:02.431334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:01:02.431334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:01:02.431334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:01:02.431334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:01:02.431334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:01:02.431334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:01:02.431334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:01:02.431334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:01:02.431334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:01:02.431334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:01:02.431334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:01:02.431334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:01:02.431334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:01:02.432491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:01:02.432491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:01:02.432491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:01:02.432491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:01:02.432491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:01:02.432491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:01:02.432491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:01:02.432491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:01:02.432491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:01:02.432491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:01:02.432491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:01:02.432491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 09:01:02.432491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:01:02.432491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:01:02.432491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:01:02.432491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:01:02.433692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:01:02.433692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:01:02.433692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:01:02.433692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:01:02.433692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:01:02.433692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:01:02.433692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:01:02.433692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:01:02.433692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:01:02.433692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:01:02.433692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:01:02.433692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 09:01:02.433692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 629/1249] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 09:01:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:01:02.439454 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624500c-dc6c-45a7-a307-da4e22f14666 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:02.550765 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624500c-dc6c-45a7-a307-da4e22f14666 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:02.588984 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:02.596442 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0481cb-e780-40cd-b311-1b3bdb787ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:02.596909 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0481cb-e780-40cd-b311-1b3bdb787ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:02.597253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0481cb-e780-40cd-b311-1b3bdb787ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:02.597540 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0481cb-e780-40cd-b311-1b3bdb787ea5 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:02.598263 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0481cb-e780-40cd-b311-1b3bdb787ea5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:02.598576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0481cb-e780-40cd-b311-1b3bdb787ea5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:02.600198 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0481cb-e780-40cd-b311-1b3bdb787ea5 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:02.609816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 621/1250] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 09:01:02 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:01:02.615875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c132d5-5f9c-4fa5-8d19-ebc03edc6d1b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:02.708312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:02.734411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c132d5-5f9c-4fa5-8d19-ebc03edc6d1b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:02.778061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804e6c4-a71c-44a9-bb2f-00d7ae78183f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:02.778916 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804e6c4-a71c-44a9-bb2f-00d7ae78183f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:02.779867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804e6c4-a71c-44a9-bb2f-00d7ae78183f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:02.780229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804e6c4-a71c-44a9-bb2f-00d7ae78183f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:02.792513 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804e6c4-a71c-44a9-bb2f-00d7ae78183f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:02.794150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804e6c4-a71c-44a9-bb2f-00d7ae78183f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:02.813846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-2804e6c4-a71c-44a9-bb2f-00d7ae78183f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:01:02.813846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:02.813846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:01:02.813846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:01:02.813846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:01:02.813846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:01:02.813846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:01:02.813846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:01:02.813846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:01:02.813846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:01:02.813846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:01:02.813846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:01:02.813846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:01:02.813846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:01:02.813846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:01:02.814930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:01:02.814930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:01:02.814930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:01:02.814930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:01:02.814930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:01:02.814930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:01:02.814930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:01:02.814930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:01:02.814930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 09:01:02.814930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:01:02.814930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:01:02.814930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:01:02.814930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:01:02.814930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:01:02.814930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:01:02.814930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:01:02.814930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:01:02.815955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:01:02.815955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:01:02.815955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:01:02.815955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 09:01:02.815955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 630/1251] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 09:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:01:02.821815 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0481cb-e780-40cd-b311-1b3bdb787ea5 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:02.941029 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0481cb-e780-40cd-b311-1b3bdb787ea5 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:02.978391 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612bfd7b-8f70-48ad-923c-6dd7e3475d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:02.979030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bfd7b-8f70-48ad-923c-6dd7e3475d34 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:02.979325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bfd7b-8f70-48ad-923c-6dd7e3475d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:02.979601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bfd7b-8f70-48ad-923c-6dd7e3475d34 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:02.980310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612bfd7b-8f70-48ad-923c-6dd7e3475d34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:02.980614 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612bfd7b-8f70-48ad-923c-6dd7e3475d34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:02.983147 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612bfd7b-8f70-48ad-923c-6dd7e3475d34 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:02.996168 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 622/1252] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 09:01:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:01:03.003510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804e6c4-a71c-44a9-bb2f-00d7ae78183f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:03.112747 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804e6c4-a71c-44a9-bb2f-00d7ae78183f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:03.141276 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:03.150473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffcca1f-9f50-44fc-9188-5b9e9da2dadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:03.151126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcca1f-9f50-44fc-9188-5b9e9da2dadd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:03.151413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcca1f-9f50-44fc-9188-5b9e9da2dadd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:03.151696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcca1f-9f50-44fc-9188-5b9e9da2dadd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:03.160670 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffcca1f-9f50-44fc-9188-5b9e9da2dadd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:03.162135 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffcca1f-9f50-44fc-9188-5b9e9da2dadd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:03.174150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-dffcca1f-9f50-44fc-9188-5b9e9da2dadd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:01:03.174150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:03.174150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:01:03.174150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:01:03.174150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:01:03.174150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:01:03.174150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:01:03.174150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:01:03.174150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:01:03.174150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:01:03.174150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:01:03.174150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:01:03.174150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:01:03.174150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:01:03.174150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:01:03.174150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:01:03.175539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:01:03.175539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:01:03.175539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:01:03.175539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:01:03.175539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:01:03.175539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:01:03.175539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:01:03.175539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:01:03.175539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:01:03.175539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:01:03.175539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:01:03.175539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 09:01:03.175539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:01:03.175539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:01:03.175539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:01:03.175539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:01:03.175539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:01:03.175539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:01:03.176711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:01:03.176711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:01:03.176711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:01:03.176711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:01:03.176711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:01:03.176711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:01:03.176711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:01:03.176711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:01:03.176711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:01:03.176711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 09:01:03.180186 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 631/1253] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 09:01:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:01:03.181094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612bfd7b-8f70-48ad-923c-6dd7e3475d34 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:03.246300 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:03.299336 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612bfd7b-8f70-48ad-923c-6dd7e3475d34 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:03.348961 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b969f-23be-4aa6-ab1d-f1890441864d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:03.349896 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b969f-23be-4aa6-ab1d-f1890441864d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:03.350230 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b969f-23be-4aa6-ab1d-f1890441864d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:03.350684 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b969f-23be-4aa6-ab1d-f1890441864d None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:03.351634 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b969f-23be-4aa6-ab1d-f1890441864d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:03.352052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b969f-23be-4aa6-ab1d-f1890441864d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:03.353945 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b969f-23be-4aa6-ab1d-f1890441864d None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:03.378611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 623/1254] 10.222.0.100 () {56 vars in 1148 bytes} [Tue Aug 2 09:01:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:01:03.384932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffcca1f-9f50-44fc-9188-5b9e9da2dadd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:03.511624 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffcca1f-9f50-44fc-9188-5b9e9da2dadd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:03.553144 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff8f2b-a61b-4a18-acbb-a80d495a654e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:03.553841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff8f2b-a61b-4a18-acbb-a80d495a654e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:03.554193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff8f2b-a61b-4a18-acbb-a80d495a654e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:03.554539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff8f2b-a61b-4a18-acbb-a80d495a654e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:03.624082 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:03.640634 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff8f2b-a61b-4a18-acbb-a80d495a654e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:03.642022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff8f2b-a61b-4a18-acbb-a80d495a654e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:03.759071 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 632/1255] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 09:01:03 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:01:03.765213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b969f-23be-4aa6-ab1d-f1890441864d None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:03.875968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b969f-23be-4aa6-ab1d-f1890441864d None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:03.892051 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:03.915013 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3600c48-5253-4995-880a-adafefae6125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:03.915668 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3600c48-5253-4995-880a-adafefae6125 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:03.916025 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3600c48-5253-4995-880a-adafefae6125 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:03.916255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3600c48-5253-4995-880a-adafefae6125 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:03.926855 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3600c48-5253-4995-880a-adafefae6125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:03.928352 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3600c48-5253-4995-880a-adafefae6125 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:03.944717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-f3600c48-5253-4995-880a-adafefae6125 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:01:03.944717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:03.944717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:01:03.944717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:01:03.944717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:01:03.944717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:01:03.944717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:01:03.944717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:01:03.944717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:01:03.944717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:01:03.944717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:01:03.944717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:01:03.944717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:01:03.944717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:01:03.944717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:01:03.946160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:01:03.946160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:01:03.946160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:01:03.946160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:01:03.946160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:01:03.946160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:01:03.946160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:01:03.946160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:01:03.946160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 09:01:03.946160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:01:03.946160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:01:03.946160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:01:03.946160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:01:03.946160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:01:03.946160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:01:03.946160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:01:03.946160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:01:03.947281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:01:03.947281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:01:03.947281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:01:03.947281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:01:03.947281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 624/1256] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 09:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:01:03.952637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff8f2b-a61b-4a18-acbb-a80d495a654e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:04.056593 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:04.070911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff8f2b-a61b-4a18-acbb-a80d495a654e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:04.105583 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967d0cb-4c69-4e40-a3ca-5a419fe427c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:04.106199 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967d0cb-4c69-4e40-a3ca-5a419fe427c3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:04.106477 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967d0cb-4c69-4e40-a3ca-5a419fe427c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:04.106779 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967d0cb-4c69-4e40-a3ca-5a419fe427c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:04.107452 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967d0cb-4c69-4e40-a3ca-5a419fe427c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:04.107751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967d0cb-4c69-4e40-a3ca-5a419fe427c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:04.109091 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967d0cb-4c69-4e40-a3ca-5a419fe427c3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:04.119917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 633/1257] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 09:01:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:01:04.126660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3600c48-5253-4995-880a-adafefae6125 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:04.248974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3600c48-5253-4995-880a-adafefae6125 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:04.292656 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278bf09-441d-42dc-bfbd-16d5f5708995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:04.293248 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278bf09-441d-42dc-bfbd-16d5f5708995 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:04.293537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278bf09-441d-42dc-bfbd-16d5f5708995 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:04.293973 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278bf09-441d-42dc-bfbd-16d5f5708995 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:04.302289 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278bf09-441d-42dc-bfbd-16d5f5708995 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:04.303928 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278bf09-441d-42dc-bfbd-16d5f5708995 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:04.316648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-d278bf09-441d-42dc-bfbd-16d5f5708995 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:01:04.316648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:04.316648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:01:04.316648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:01:04.316648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:01:04.316648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:01:04.316648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:01:04.316648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:01:04.316648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:01:04.316648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:01:04.316648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:01:04.316648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:01:04.316648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:01:04.316648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:01:04.316648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:01:04.317791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:01:04.317791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:01:04.317791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:01:04.317791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:01:04.317791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:01:04.317791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:01:04.317791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:01:04.317791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:01:04.317791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 09:01:04.317791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:01:04.317791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:01:04.317791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:01:04.317791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:01:04.317791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:01:04.317791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:01:04.317791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:01:04.317791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:01:04.318859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:01:04.318859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:01:04.318859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:01:04.318859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:01:04.318859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:01:04.318859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:01:04.318859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:01:04.318859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:01:04.318859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 625/1258] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 09:01:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:01:04.323950 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967d0cb-4c69-4e40-a3ca-5a419fe427c3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:04.441320 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967d0cb-4c69-4e40-a3ca-5a419fe427c3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:04.481197 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f8ac7-0428-4714-a512-17cded6db61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:04.481979 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f8ac7-0428-4714-a512-17cded6db61f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:04.482293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f8ac7-0428-4714-a512-17cded6db61f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:04.482643 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f8ac7-0428-4714-a512-17cded6db61f None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:04.483469 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f8ac7-0428-4714-a512-17cded6db61f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:04.483848 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f8ac7-0428-4714-a512-17cded6db61f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:04.485372 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f8ac7-0428-4714-a512-17cded6db61f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:04.499682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 634/1259] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 09:01:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:01:04.505976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278bf09-441d-42dc-bfbd-16d5f5708995 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:04.536984 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:04.622829 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278bf09-441d-42dc-bfbd-16d5f5708995 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:04.659799 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b10c7-5db9-40f2-a8a9-b7377b9296bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:04.660414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b10c7-5db9-40f2-a8a9-b7377b9296bc None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:04.660695 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b10c7-5db9-40f2-a8a9-b7377b9296bc None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:04.660972 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b10c7-5db9-40f2-a8a9-b7377b9296bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:04.661788 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b10c7-5db9-40f2-a8a9-b7377b9296bc None admin] RBAC: Adding query filter params (role.id=7cbff51867a7475d837afd51f6c12f11, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=b40981fe179a43dc81553d3594dad649) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:04.662103 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b10c7-5db9-40f2-a8a9-b7377b9296bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:04.663571 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b10c7-5db9-40f2-a8a9-b7377b9296bc None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:04.669677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 626/1260] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 09:01:04 2022] GET /identity/v3/role_assignments?user.id=b40981fe179a43dc81553d3594dad649&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=7cbff51867a7475d837afd51f6c12f11 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:01:06.800950 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b83c7-f2e9-42a5-ba29-39b14ca48340 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:06.801537 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b83c7-f2e9-42a5-ba29-39b14ca48340 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:06.801537 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b83c7-f2e9-42a5-ba29-39b14ca48340 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:06.802917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 635/1261] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:01:06 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:01:06.808142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f7e79-8cc7-4418-a892-47e168537fcb None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:06.808551 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f7e79-8cc7-4418-a892-47e168537fcb None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:06.808707 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f7e79-8cc7-4418-a892-47e168537fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:06.827375 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-558f7e79-8cc7-4418-a892-47e168537fcb None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:01:06.828562 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-558f7e79-8cc7-4418-a892-47e168537fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:01:06.894316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 627/1262] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:01:06 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:01:06.920628 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b83c7-f2e9-42a5-ba29-39b14ca48340 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:06.931683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:07.021614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b83c7-f2e9-42a5-ba29-39b14ca48340 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:07.056308 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a6b75-cc4b-4133-9459-7596492a8b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:07.056883 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a6b75-cc4b-4133-9459-7596492a8b9b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:07.057194 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a6b75-cc4b-4133-9459-7596492a8b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:07.057476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a6b75-cc4b-4133-9459-7596492a8b9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:07.067320 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a6b75-cc4b-4133-9459-7596492a8b9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:01:07.072234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-db1a6b75-cc4b-4133-9459-7596492a8b9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:01:07.072234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:07.072234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:01:07.072234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:01:07.072234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:01:07.072234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:01:07.072234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:01:07.072234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:01:07.072234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:01:07.072234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:01:07.072234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:01:07.072234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:01:07.072234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:01:07.072234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:01:07.072234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:01:07.072234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:01:07.073391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:01:07.073391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:01:07.073391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:01:07.073391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:01:07.073391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:01:07.073391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:01:07.073391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:01:07.073391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:01:07.073391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:01:07.073391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:01:07.073391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:01:07.073391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 09:01:07.073391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:01:07.073391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:01:07.073391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:01:07.073391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:01:07.074632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:01:07.074632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:01:07.074632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:01:07.074632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:01:07.074632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:01:07.074632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:01:07.074632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:01:07.074632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:01:07.074632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:01:07.074632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:01:07.074632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:01:07.074632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 09:01:07.074632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 636/1263] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 09:01:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:01:07.079460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f7e79-8cc7-4418-a892-47e168537fcb None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:07.097965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:07.177449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f7e79-8cc7-4418-a892-47e168537fcb None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:07.209888 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b07d02-4c80-45d7-b803-880bec8a73d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:07.210518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b07d02-4c80-45d7-b803-880bec8a73d7 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:07.210802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b07d02-4c80-45d7-b803-880bec8a73d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:07.211079 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b07d02-4c80-45d7-b803-880bec8a73d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:07.211784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b07d02-4c80-45d7-b803-880bec8a73d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:07.212090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b07d02-4c80-45d7-b803-880bec8a73d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:07.213499 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b07d02-4c80-45d7-b803-880bec8a73d7 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:07.221883 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 628/1264] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 09:01:07 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:01:07.227565 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a6b75-cc4b-4133-9459-7596492a8b9b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:07.292921 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:07.326507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a6b75-cc4b-4133-9459-7596492a8b9b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:07.359863 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5047a-4df5-4421-8bb5-22e6ff0a6b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:07.360497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5047a-4df5-4421-8bb5-22e6ff0a6b60 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:07.360786 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5047a-4df5-4421-8bb5-22e6ff0a6b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:07.361063 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5047a-4df5-4421-8bb5-22e6ff0a6b60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:07.371587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5047a-4df5-4421-8bb5-22e6ff0a6b60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:07.373043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5047a-4df5-4421-8bb5-22e6ff0a6b60 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:07.386944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-caf5047a-4df5-4421-8bb5-22e6ff0a6b60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:01:07.386944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:07.386944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:01:07.386944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:01:07.386944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:01:07.386944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:01:07.386944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:01:07.386944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:01:07.386944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:01:07.386944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:01:07.386944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:01:07.386944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:01:07.386944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:01:07.386944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:01:07.386944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:01:07.388538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:01:07.388538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:01:07.388538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:01:07.388538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:01:07.388538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:01:07.388538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:01:07.388538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:01:07.388538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:01:07.388538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 09:01:07.388538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:01:07.388538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:01:07.388538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:01:07.388538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:01:07.388538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:01:07.388538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:01:07.388538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:01:07.388538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:01:07.389798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:01:07.389798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:01:07.389798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:01:07.389798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 09:01:07.389798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 637/1265] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 09:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:01:07.394229 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b07d02-4c80-45d7-b803-880bec8a73d7 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:07.473765 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:07.492066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b07d02-4c80-45d7-b803-880bec8a73d7 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:07.525546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a985f3a-3ab5-4041-9690-fa5493e377fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:07.526156 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a985f3a-3ab5-4041-9690-fa5493e377fc None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:07.526436 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a985f3a-3ab5-4041-9690-fa5493e377fc None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:07.526707 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a985f3a-3ab5-4041-9690-fa5493e377fc None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:07.527381 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a985f3a-3ab5-4041-9690-fa5493e377fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:07.527683 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a985f3a-3ab5-4041-9690-fa5493e377fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:07.529648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a985f3a-3ab5-4041-9690-fa5493e377fc None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:07.540997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 629/1266] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 09:01:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:01:07.546666 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5047a-4df5-4421-8bb5-22e6ff0a6b60 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:07.662892 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5047a-4df5-4421-8bb5-22e6ff0a6b60 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:07.681977 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:07.695944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff8c16-ba8b-41da-9070-494216f09f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:07.696584 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff8c16-ba8b-41da-9070-494216f09f02 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:07.696871 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff8c16-ba8b-41da-9070-494216f09f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:07.697179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff8c16-ba8b-41da-9070-494216f09f02 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:07.705044 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff8c16-ba8b-41da-9070-494216f09f02 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:07.706485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff8c16-ba8b-41da-9070-494216f09f02 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:07.718203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-0cff8c16-ba8b-41da-9070-494216f09f02 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:01:07.718203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:07.718203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:01:07.718203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:01:07.718203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:01:07.718203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:01:07.718203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:01:07.718203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:01:07.718203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:01:07.718203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:01:07.718203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:01:07.718203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:01:07.718203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:01:07.718203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:01:07.718203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:01:07.718203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:01:07.719263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:01:07.719263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:01:07.719263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:01:07.719263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:01:07.719263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:01:07.719263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:01:07.719263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:01:07.719263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:01:07.719263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:01:07.719263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:01:07.719263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:01:07.719263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 09:01:07.719263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:01:07.719263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:01:07.719263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:01:07.719263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:01:07.719263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:01:07.719263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:01:07.720527 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:01:07.720527 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:01:07.720527 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:01:07.720527 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:01:07.720527 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:01:07.720527 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:01:07.720527 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:01:07.720527 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:01:07.720527 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:01:07.720527 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 09:01:07.720527 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 638/1267] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 09:01:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:01:07.725180 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a985f3a-3ab5-4041-9690-fa5493e377fc None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:07.820475 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a985f3a-3ab5-4041-9690-fa5493e377fc None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:07.853085 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dda4e0-f63d-425f-8e66-0c47fe8566b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:07.853717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dda4e0-f63d-425f-8e66-0c47fe8566b6 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:07.854003 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dda4e0-f63d-425f-8e66-0c47fe8566b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:07.854287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dda4e0-f63d-425f-8e66-0c47fe8566b6 None admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:07.855070 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dda4e0-f63d-425f-8e66-0c47fe8566b6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:07.855382 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dda4e0-f63d-425f-8e66-0c47fe8566b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:07.857004 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dda4e0-f63d-425f-8e66-0c47fe8566b6 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:07.865592 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:07.874988 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 630/1268] 10.222.0.100 () {56 vars in 1148 bytes} [Tue Aug 2 09:01:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:01:07.880590 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff8c16-ba8b-41da-9070-494216f09f02 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:07.978800 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff8c16-ba8b-41da-9070-494216f09f02 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:08.011693 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00eba0-6014-4df6-ae56-8c61f3dc4aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:08.012326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00eba0-6014-4df6-ae56-8c61f3dc4aff None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:08.012609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00eba0-6014-4df6-ae56-8c61f3dc4aff None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:08.013490 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00eba0-6014-4df6-ae56-8c61f3dc4aff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:08.023609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00eba0-6014-4df6-ae56-8c61f3dc4aff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:08.025025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00eba0-6014-4df6-ae56-8c61f3dc4aff None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:08.038983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-4c00eba0-6014-4df6-ae56-8c61f3dc4aff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:01:08.038983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:08.038983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:01:08.038983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:01:08.038983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:01:08.038983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:01:08.038983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:01:08.038983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:01:08.038983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:01:08.038983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:01:08.038983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:01:08.038983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:01:08.038983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:01:08.038983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:01:08.038983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:01:08.040024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:01:08.040024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:01:08.040024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:01:08.040024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:01:08.040024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:01:08.040024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:01:08.040024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:01:08.040024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:01:08.040024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 09:01:08.040024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:01:08.040024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:01:08.040024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:01:08.040024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:01:08.040024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:01:08.040024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:01:08.040024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:01:08.040024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:01:08.040967 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:01:08.040967 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:01:08.040967 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:01:08.040967 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 09:01:08.040967 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 639/1269] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 09:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:01:08.045910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dda4e0-f63d-425f-8e66-0c47fe8566b6 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:08.142666 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dda4e0-f63d-425f-8e66-0c47fe8566b6 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:08.175236 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf838022-8ff9-493f-9c13-cd48d13f8dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:08.175780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf838022-8ff9-493f-9c13-cd48d13f8dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:08.176060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf838022-8ff9-493f-9c13-cd48d13f8dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:08.176331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf838022-8ff9-493f-9c13-cd48d13f8dd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:08.177004 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf838022-8ff9-493f-9c13-cd48d13f8dd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:08.177339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf838022-8ff9-493f-9c13-cd48d13f8dd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:08.178695 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf838022-8ff9-493f-9c13-cd48d13f8dd3 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:08.190117 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 631/1270] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 09:01:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:01:08.195519 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00eba0-6014-4df6-ae56-8c61f3dc4aff None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:08.227199 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:08.295158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00eba0-6014-4df6-ae56-8c61f3dc4aff None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:08.327745 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8c5e9-6114-4b6f-9d37-8a8df31414ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:08.328363 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8c5e9-6114-4b6f-9d37-8a8df31414ee None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:08.328644 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8c5e9-6114-4b6f-9d37-8a8df31414ee None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:08.328917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8c5e9-6114-4b6f-9d37-8a8df31414ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:08.335381 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8c5e9-6114-4b6f-9d37-8a8df31414ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:08.336792 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8c5e9-6114-4b6f-9d37-8a8df31414ee None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:08.349049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-4ba8c5e9-6114-4b6f-9d37-8a8df31414ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:01:08.349049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:08.349049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:01:08.349049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:01:08.349049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:01:08.349049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:01:08.349049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:01:08.349049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:01:08.349049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:01:08.349049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:01:08.349049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:01:08.349049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:01:08.349049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:01:08.349049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:01:08.349049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:01:08.350099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:01:08.350099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:01:08.350099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:01:08.350099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:01:08.350099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:01:08.350099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:01:08.350099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:01:08.350099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:01:08.350099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 09:01:08.350099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:01:08.350099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:01:08.350099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:01:08.350099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:01:08.350099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:01:08.350099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:01:08.350099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:01:08.350099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:01:08.351030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:01:08.351030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:01:08.351030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:01:08.351030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:01:08.351030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:01:08.351030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:01:08.351030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:01:08.351030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 09:01:08.351030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 640/1271] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 09:01:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:01:08.355923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf838022-8ff9-493f-9c13-cd48d13f8dd3 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:08.412348 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:08.453262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf838022-8ff9-493f-9c13-cd48d13f8dd3 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:08.485003 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ab320-c27b-4218-abf2-c6bcd0f6f60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:08.485627 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ab320-c27b-4218-abf2-c6bcd0f6f60c None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:08.485907 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ab320-c27b-4218-abf2-c6bcd0f6f60c None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:08.486180 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ab320-c27b-4218-abf2-c6bcd0f6f60c None admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:08.486907 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ab320-c27b-4218-abf2-c6bcd0f6f60c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:08.487208 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ab320-c27b-4218-abf2-c6bcd0f6f60c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:08.488623 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ab320-c27b-4218-abf2-c6bcd0f6f60c None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:08.500919 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 632/1272] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 09:01:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:01:08.506969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8c5e9-6114-4b6f-9d37-8a8df31414ee None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:08.600782 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:08.609838 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8c5e9-6114-4b6f-9d37-8a8df31414ee None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:08.643488 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9ac40-5a90-4b97-97f3-21b8c004f500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:08.644143 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9ac40-5a90-4b97-97f3-21b8c004f500 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:08.644419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9ac40-5a90-4b97-97f3-21b8c004f500 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:08.644688 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9ac40-5a90-4b97-97f3-21b8c004f500 None admin] PATH_INFO: `/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/b40981fe179a43dc81553d3594dad649/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:08.646587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9ac40-5a90-4b97-97f3-21b8c004f500 None admin] RBAC: Authorizing `identity:create_grant(project_id=61ca0176324e445b94e42a5214d2c89b, user_id=b40981fe179a43dc81553d3594dad649, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:08.647989 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9ac40-5a90-4b97-97f3-21b8c004f500 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:08.659937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 641/1273] 10.222.0.100 () {58 vars in 1444 bytes} [Tue Aug 2 09:01:08 2022] PUT /identity/v3/projects/61ca0176324e445b94e42a5214d2c89b/users/b40981fe179a43dc81553d3594dad649/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:01:10.610574 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aee93a-3cb2-4a04-9dde-da8046e01c48 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:10.611138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aee93a-3cb2-4a04-9dde-da8046e01c48 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:10.611138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aee93a-3cb2-4a04-9dde-da8046e01c48 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:10.612474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 633/1274] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:01:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:01:10.617522 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef45d4-8fd6-458e-ae2a-58db63fbf1f5 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:10.617953 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef45d4-8fd6-458e-ae2a-58db63fbf1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:10.618091 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef45d4-8fd6-458e-ae2a-58db63fbf1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:10.636163 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f0ef45d4-8fd6-458e-ae2a-58db63fbf1f5 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:01:10.637377 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f0ef45d4-8fd6-458e-ae2a-58db63fbf1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:01:10.703197 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 642/1275] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:01:10 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:01:10.726321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aee93a-3cb2-4a04-9dde-da8046e01c48 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:10.825679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aee93a-3cb2-4a04-9dde-da8046e01c48 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:10.842008 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:10.858234 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b40d63-59f5-4dae-b14b-ee3c275854c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:10.858853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b40d63-59f5-4dae-b14b-ee3c275854c6 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:10.859130 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b40d63-59f5-4dae-b14b-ee3c275854c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:10.859400 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b40d63-59f5-4dae-b14b-ee3c275854c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:10.869066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b40d63-59f5-4dae-b14b-ee3c275854c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:01:10.874977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-41b40d63-59f5-4dae-b14b-ee3c275854c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:01:10.874977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:10.874977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:01:10.874977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:01:10.874977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:01:10.874977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:01:10.874977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:01:10.874977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:01:10.874977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:01:10.874977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:01:10.874977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:01:10.874977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:01:10.874977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:01:10.874977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:01:10.874977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:01:10.874977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:01:10.876073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:01:10.876073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:01:10.876073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:01:10.876073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:01:10.876073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:01:10.876073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:01:10.876073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:01:10.876073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:01:10.876073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:01:10.876073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:01:10.876073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:01:10.876073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 09:01:10.876073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:01:10.876073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:01:10.876073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:01:10.876073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:01:10.876942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:01:10.876942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:01:10.876942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:01:10.876942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:01:10.876942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:01:10.876942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:01:10.876942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:01:10.876942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:01:10.876942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:01:10.876942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:01:10.876942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:01:10.876942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:01:10.876942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 634/1276] 10.222.0.100 () {56 vars in 1101 bytes} [Tue Aug 2 09:01:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:01:10.882034 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef45d4-8fd6-458e-ae2a-58db63fbf1f5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:10.962792 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:10.982406 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef45d4-8fd6-458e-ae2a-58db63fbf1f5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:11.015684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ab532-17fa-4df5-a936-8f0b38273437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:11.016296 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ab532-17fa-4df5-a936-8f0b38273437 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:11.016575 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ab532-17fa-4df5-a936-8f0b38273437 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:11.016847 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ab532-17fa-4df5-a936-8f0b38273437 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:11.017573 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ab532-17fa-4df5-a936-8f0b38273437 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:11.017876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ab532-17fa-4df5-a936-8f0b38273437 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:11.019218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ab532-17fa-4df5-a936-8f0b38273437 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:11.026963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 643/1277] 10.222.0.100 () {56 vars in 1104 bytes} [Tue Aug 2 09:01:10 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:01:11.032772 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b40d63-59f5-4dae-b14b-ee3c275854c6 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:11.129506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b40d63-59f5-4dae-b14b-ee3c275854c6 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:11.162236 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92599c-e7ea-4b19-8025-c4aff7e60458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:11.162863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92599c-e7ea-4b19-8025-c4aff7e60458 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:11.163142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92599c-e7ea-4b19-8025-c4aff7e60458 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:11.163416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92599c-e7ea-4b19-8025-c4aff7e60458 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:11.173652 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f92599c-e7ea-4b19-8025-c4aff7e60458 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:11.175089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f92599c-e7ea-4b19-8025-c4aff7e60458 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:11.189929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-2f92599c-e7ea-4b19-8025-c4aff7e60458 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:01:11.189929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:11.189929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:01:11.189929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:01:11.189929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:01:11.189929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:01:11.189929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:01:11.189929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:01:11.189929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:01:11.189929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:01:11.189929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:01:11.189929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:01:11.189929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:01:11.189929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:01:11.189929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:01:11.190976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:01:11.190976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:01:11.190976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:01:11.190976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:01:11.190976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:01:11.190976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:01:11.190976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:01:11.190976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:01:11.190976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 09:01:11.190976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:01:11.190976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:01:11.190976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:01:11.190976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:01:11.190976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:01:11.190976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:01:11.190976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:01:11.190976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:01:11.192089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:01:11.192089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:01:11.192089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:01:11.192089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:01:11.192089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 635/1278] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 09:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:01:11.196645 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ab532-17fa-4df5-a936-8f0b38273437 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:11.293520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ab532-17fa-4df5-a936-8f0b38273437 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:11.326370 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bb2fc-e6e0-4353-afec-e4baa8470fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:11.326973 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bb2fc-e6e0-4353-afec-e4baa8470fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:11.327258 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bb2fc-e6e0-4353-afec-e4baa8470fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:11.327536 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bb2fc-e6e0-4353-afec-e4baa8470fb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:11.328218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2bb2fc-e6e0-4353-afec-e4baa8470fb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:11.328527 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2bb2fc-e6e0-4353-afec-e4baa8470fb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:11.329905 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2bb2fc-e6e0-4353-afec-e4baa8470fb2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:11.341436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 644/1279] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 09:01:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:01:11.347084 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92599c-e7ea-4b19-8025-c4aff7e60458 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:11.378048 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:11.445608 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92599c-e7ea-4b19-8025-c4aff7e60458 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:11.477262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a5afe-cfb4-4e45-a30f-0eb38463e393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:11.477911 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a5afe-cfb4-4e45-a30f-0eb38463e393 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:11.478189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a5afe-cfb4-4e45-a30f-0eb38463e393 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:11.478458 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a5afe-cfb4-4e45-a30f-0eb38463e393 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:11.486554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a5afe-cfb4-4e45-a30f-0eb38463e393 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:11.487964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a5afe-cfb4-4e45-a30f-0eb38463e393 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:11.499882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-1f5a5afe-cfb4-4e45-a30f-0eb38463e393 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:01:11.499882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:11.499882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:01:11.499882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:01:11.499882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:01:11.499882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:01:11.499882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:01:11.499882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:01:11.499882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:01:11.499882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:01:11.499882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:01:11.499882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:01:11.499882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:01:11.499882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:01:11.499882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:01:11.499882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:01:11.501007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:01:11.501007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:01:11.501007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:01:11.501007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:01:11.501007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:01:11.501007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:01:11.501007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:01:11.501007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:01:11.501007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:01:11.501007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:01:11.501007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:01:11.501007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 09:01:11.501007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:01:11.501007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:01:11.501007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:01:11.501007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:01:11.501007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:01:11.501007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:01:11.502183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:01:11.502183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:01:11.502183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:01:11.502183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:01:11.502183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:01:11.502183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:01:11.502183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:01:11.502183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:01:11.502183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:01:11.502183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:01:11.502183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 636/1280] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 09:01:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:01:11.506903 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bb2fc-e6e0-4353-afec-e4baa8470fb2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:11.510573 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:11.604113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bb2fc-e6e0-4353-afec-e4baa8470fb2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:11.637201 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d4cd1-a069-4330-91f7-14292aeb3039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:11.637847 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d4cd1-a069-4330-91f7-14292aeb3039 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:11.638126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d4cd1-a069-4330-91f7-14292aeb3039 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:11.638396 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d4cd1-a069-4330-91f7-14292aeb3039 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:11.639168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d4cd1-a069-4330-91f7-14292aeb3039 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:11.639469 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d4cd1-a069-4330-91f7-14292aeb3039 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:11.640864 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d4cd1-a069-4330-91f7-14292aeb3039 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:11.658480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 645/1281] 10.222.0.100 () {56 vars in 1148 bytes} [Tue Aug 2 09:01:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:01:11.664361 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a5afe-cfb4-4e45-a30f-0eb38463e393 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:11.754422 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:11.764448 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a5afe-cfb4-4e45-a30f-0eb38463e393 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:11.797102 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abeea97-80c0-4e6c-8e50-65079dc32ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:11.797728 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeea97-80c0-4e6c-8e50-65079dc32ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:11.798007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeea97-80c0-4e6c-8e50-65079dc32ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:11.798278 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeea97-80c0-4e6c-8e50-65079dc32ad2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:11.861593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abeea97-80c0-4e6c-8e50-65079dc32ad2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:11.863014 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abeea97-80c0-4e6c-8e50-65079dc32ad2 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:11.949207 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:11.960522 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 637/1282] 10.222.0.100 () {58 vars in 1287 bytes} [Tue Aug 2 09:01:11 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:01:11.967451 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d4cd1-a069-4330-91f7-14292aeb3039 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:12.029907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:12.067781 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d4cd1-a069-4330-91f7-14292aeb3039 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:12.102017 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dccfa-594a-4cc5-9408-1fa4df2bc24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:12.102648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dccfa-594a-4cc5-9408-1fa4df2bc24c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:12.102930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dccfa-594a-4cc5-9408-1fa4df2bc24c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:12.103206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dccfa-594a-4cc5-9408-1fa4df2bc24c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:12.113375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8dccfa-594a-4cc5-9408-1fa4df2bc24c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:12.114790 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8dccfa-594a-4cc5-9408-1fa4df2bc24c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:12.129741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-bc8dccfa-594a-4cc5-9408-1fa4df2bc24c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:01:12.129741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:12.129741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:01:12.129741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:01:12.129741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:01:12.129741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:01:12.129741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:01:12.129741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:01:12.129741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:01:12.129741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:01:12.129741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:01:12.129741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:01:12.129741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:01:12.129741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:01:12.129741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:01:12.130772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:01:12.130772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:01:12.130772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:01:12.130772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:01:12.130772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:01:12.130772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:01:12.130772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:01:12.130772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:01:12.130772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 09:01:12.130772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:01:12.130772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:01:12.130772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:01:12.130772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:01:12.130772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:01:12.130772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:01:12.130772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:01:12.130772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:01:12.131730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:01:12.131730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:01:12.131730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:01:12.131730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 09:01:12.131730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 646/1283] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 09:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:01:12.136518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abeea97-80c0-4e6c-8e50-65079dc32ad2 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:12.232612 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abeea97-80c0-4e6c-8e50-65079dc32ad2 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:12.266018 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaeb647-407e-47c9-b64d-57f08bc290b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:12.266624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaeb647-407e-47c9-b64d-57f08bc290b6 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:12.266904 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaeb647-407e-47c9-b64d-57f08bc290b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:12.267178 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaeb647-407e-47c9-b64d-57f08bc290b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:12.267881 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaeb647-407e-47c9-b64d-57f08bc290b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:12.268185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaeb647-407e-47c9-b64d-57f08bc290b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:12.269571 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaeb647-407e-47c9-b64d-57f08bc290b6 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:12.281225 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 638/1284] 10.222.0.100 () {56 vars in 1114 bytes} [Tue Aug 2 09:01:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:01:12.286496 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dccfa-594a-4cc5-9408-1fa4df2bc24c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:12.382824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dccfa-594a-4cc5-9408-1fa4df2bc24c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:12.396451 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:12.416543 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6dcdf7-5fa1-4caa-bdd7-736f587777d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:12.417208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dcdf7-5fa1-4caa-bdd7-736f587777d2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:12.417494 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dcdf7-5fa1-4caa-bdd7-736f587777d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:12.417773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dcdf7-5fa1-4caa-bdd7-736f587777d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:12.424432 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6dcdf7-5fa1-4caa-bdd7-736f587777d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:12.425876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6dcdf7-5fa1-4caa-bdd7-736f587777d2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:12.438126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-6c6dcdf7-5fa1-4caa-bdd7-736f587777d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:01:12.438126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:12.438126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:01:12.438126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:01:12.438126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:01:12.438126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:01:12.438126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:01:12.438126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:01:12.438126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:01:12.438126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:01:12.438126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:01:12.438126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:01:12.438126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:01:12.438126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:01:12.438126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:01:12.439167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:01:12.439167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:01:12.439167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:01:12.439167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:01:12.439167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:01:12.439167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:01:12.439167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:01:12.439167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:01:12.439167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Aug 02 09:01:12.439167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:01:12.439167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:01:12.439167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:01:12.439167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:01:12.439167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:01:12.439167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:01:12.439167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:01:12.439167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:01:12.440294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:01:12.440294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:01:12.440294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:01:12.440294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:01:12.440294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:01:12.440294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:01:12.440294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:01:12.440294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Aug 02 09:01:12.440294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 647/1285] 10.222.0.100 () {56 vars in 1116 bytes} [Tue Aug 2 09:01:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:01:12.445012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaeb647-407e-47c9-b64d-57f08bc290b6 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:12.493360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:12.542630 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaeb647-407e-47c9-b64d-57f08bc290b6 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:12.574799 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa52db-98ef-4ceb-8f66-f7ce5587baf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:12.576112 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa52db-98ef-4ceb-8f66-f7ce5587baf8 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:12.576393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa52db-98ef-4ceb-8f66-f7ce5587baf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:12.576667 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa52db-98ef-4ceb-8f66-f7ce5587baf8 None admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:12.577429 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa52db-98ef-4ceb-8f66-f7ce5587baf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:12.577735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa52db-98ef-4ceb-8f66-f7ce5587baf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:12.579150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa52db-98ef-4ceb-8f66-f7ce5587baf8 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:12.590777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 639/1286] 10.222.0.100 () {56 vars in 1153 bytes} [Tue Aug 2 09:01:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:01:12.596649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6dcdf7-5fa1-4caa-bdd7-736f587777d2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:12.696283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6dcdf7-5fa1-4caa-bdd7-736f587777d2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:12.728750 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f78acd7-e5ba-4eb4-9ef9-2b467c121976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:12.729387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78acd7-e5ba-4eb4-9ef9-2b467c121976 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:12.729674 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78acd7-e5ba-4eb4-9ef9-2b467c121976 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:12.729952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78acd7-e5ba-4eb4-9ef9-2b467c121976 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:12.730723 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f78acd7-e5ba-4eb4-9ef9-2b467c121976 None admin] RBAC: Adding query filter params (role.id=7cbff51867a7475d837afd51f6c12f11, scope.project.id=61ca0176324e445b94e42a5214d2c89b, user.id=b40981fe179a43dc81553d3594dad649) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:12.731037 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f78acd7-e5ba-4eb4-9ef9-2b467c121976 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:12.733081 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f78acd7-e5ba-4eb4-9ef9-2b467c121976 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:12.738817 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 648/1287] 10.222.0.100 () {56 vars in 1379 bytes} [Tue Aug 2 09:01:12 2022] GET /identity/v3/role_assignments?user.id=b40981fe179a43dc81553d3594dad649&scope.project.id=61ca0176324e445b94e42a5214d2c89b&role.id=7cbff51867a7475d837afd51f6c12f11 => generated 610 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:01:14.728571 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48fd41-f9e7-4d81-b276-051cadcf90b4 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:14.729097 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48fd41-f9e7-4d81-b276-051cadcf90b4 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:14.729097 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48fd41-f9e7-4d81-b276-051cadcf90b4 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:14.730504 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 640/1288] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:01:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:01:14.736346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa307e-76f0-43a3-9fe2-95ab6960cf91 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:14.736769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa307e-76f0-43a3-9fe2-95ab6960cf91 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:14.736914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa307e-76f0-43a3-9fe2-95ab6960cf91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:14.755498 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f0fa307e-76f0-43a3-9fe2-95ab6960cf91 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:01:14.756674 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f0fa307e-76f0-43a3-9fe2-95ab6960cf91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:01:14.785997 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:14.822697 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 649/1289] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:01:14 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:01:14.845340 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48fd41-f9e7-4d81-b276-051cadcf90b4 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:14.946035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48fd41-f9e7-4d81-b276-051cadcf90b4 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:14.956242 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:14.979380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb78914-7f39-465b-a8ad-52286fc075f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:14.979941 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb78914-7f39-465b-a8ad-52286fc075f2 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:14.980219 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb78914-7f39-465b-a8ad-52286fc075f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:14.980489 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb78914-7f39-465b-a8ad-52286fc075f2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:14.981805 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb78914-7f39-465b-a8ad-52286fc075f2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:14.983169 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb78914-7f39-465b-a8ad-52286fc075f2 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:14.991801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-cdb78914-7f39-465b-a8ad-52286fc075f2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 09:01:14.991801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:14.991801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:01:14.991801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:01:14.991801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:01:14.991801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:01:14.991801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:01:14.991801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:01:14.991801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 09:01:14.991801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:01:14.991801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:01:14.991801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:01:14.991801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:14.991801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:01:14.991801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:01:14.991801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:01:14.991801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:01:14.992965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:01:14.992965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:01:14.992965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:01:14.992965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:01:14.992965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:01:14.992965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:01:14.992965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:01:14.992965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:01:14.992965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:01:14.992965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:01:14.992965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:01:14.992965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:01:14.992965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:01:14.992965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:01:14.992965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:01:14.992965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:01:14.994011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:01:14.994011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 09:01:14.994011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:01:14.994011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:01:14.994011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:01:14.994011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:01:14.994011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:01:14.994011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:01:14.994011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:01:14.994011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:01:14.994011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:01:14.994011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:01:14.994011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 09:01:14.994011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:01:14.994011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 641/1290] 10.222.0.100 () {56 vars in 1122 bytes} [Tue Aug 2 09:01:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:01:14.999983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa307e-76f0-43a3-9fe2-95ab6960cf91 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:15.094567 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa307e-76f0-43a3-9fe2-95ab6960cf91 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:15.128577 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485921c9-32c8-403a-8822-ee8b4763a8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:15.129206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485921c9-32c8-403a-8822-ee8b4763a8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:15.129489 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485921c9-32c8-403a-8822-ee8b4763a8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:15.129759 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485921c9-32c8-403a-8822-ee8b4763a8b0 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:15.130422 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485921c9-32c8-403a-8822-ee8b4763a8b0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:15.130725 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485921c9-32c8-403a-8822-ee8b4763a8b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:15.132063 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485921c9-32c8-403a-8822-ee8b4763a8b0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:15.136694 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 650/1291] 10.222.0.100 () {56 vars in 1121 bytes} [Tue Aug 2 09:01:14 2022] GET /identity/v3/services?name=placement => generated 127 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:01:15.142403 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb78914-7f39-465b-a8ad-52286fc075f2 None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:15.240181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb78914-7f39-465b-a8ad-52286fc075f2 None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:15.272892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae1088-f6c7-449c-aee0-3af919237768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:15.273519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae1088-f6c7-449c-aee0-3af919237768 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:15.273834 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae1088-f6c7-449c-aee0-3af919237768 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:15.274113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae1088-f6c7-449c-aee0-3af919237768 None admin] PATH_INFO: `/v3/services` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:15.274782 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae1088-f6c7-449c-aee0-3af919237768 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:15.275092 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae1088-f6c7-449c-aee0-3af919237768 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:15.276453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae1088-f6c7-449c-aee0-3af919237768 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:15.281363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 642/1292] 10.222.0.100 () {56 vars in 1121 bytes} [Tue Aug 2 09:01:15 2022] GET /identity/v3/services?type=placement => generated 127 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:01:17.268219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06865496-bc99-49fe-8d69-9afab92a2cd4 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:17.268836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06865496-bc99-49fe-8d69-9afab92a2cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:17.268836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06865496-bc99-49fe-8d69-9afab92a2cd4 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:17.270139 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 651/1293] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:01:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:01:17.275087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6f0ff-fc38-4e33-8dc0-a0f8c6fddf55 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:17.275481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6f0ff-fc38-4e33-8dc0-a0f8c6fddf55 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:17.275654 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6f0ff-fc38-4e33-8dc0-a0f8c6fddf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:17.293921 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-31e6f0ff-fc38-4e33-8dc0-a0f8c6fddf55 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:01:17.295165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-31e6f0ff-fc38-4e33-8dc0-a0f8c6fddf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:01:17.324245 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:17.361283 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 643/1294] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:01:17 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:01:17.384057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06865496-bc99-49fe-8d69-9afab92a2cd4 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:17.415738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:17.484452 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06865496-bc99-49fe-8d69-9afab92a2cd4 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:17.517380 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a06df5-dd14-4ccb-92bb-395933ac11b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:17.517983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a06df5-dd14-4ccb-92bb-395933ac11b6 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:17.518313 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a06df5-dd14-4ccb-92bb-395933ac11b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:17.518533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a06df5-dd14-4ccb-92bb-395933ac11b6 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:17.519221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a06df5-dd14-4ccb-92bb-395933ac11b6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:17.520553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a06df5-dd14-4ccb-92bb-395933ac11b6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:17.534772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 652/1295] 10.222.0.100 () {60 vars in 1146 bytes} [Tue Aug 2 09:01:17 2022] POST /identity/v3/services => generated 247 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:01:19.541766 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a2ed1-79a7-4fff-a6f7-9f5391f8e53f None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:19.542331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a2ed1-79a7-4fff-a6f7-9f5391f8e53f None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:19.542331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a2ed1-79a7-4fff-a6f7-9f5391f8e53f None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:19.544367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 644/1296] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:01:19 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:01:19.549575 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8e4ea-63db-49bd-8c86-206689e4e666 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:19.550001 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8e4ea-63db-49bd-8c86-206689e4e666 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:19.550138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8e4ea-63db-49bd-8c86-206689e4e666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:19.568844 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-19e8e4ea-63db-49bd-8c86-206689e4e666 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:01:19.570257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-19e8e4ea-63db-49bd-8c86-206689e4e666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:01:19.664067 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 653/1297] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:01:19 2022] POST /identity/v3/auth/tokens => generated 2226 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:01:19.687423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a2ed1-79a7-4fff-a6f7-9f5391f8e53f None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:19.788276 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a2ed1-79a7-4fff-a6f7-9f5391f8e53f None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:19.821486 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0a161-2b5c-4bfc-ad4a-5c128560436d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:19.822042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0a161-2b5c-4bfc-ad4a-5c128560436d None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:19.822319 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0a161-2b5c-4bfc-ad4a-5c128560436d None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:19.822590 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0a161-2b5c-4bfc-ad4a-5c128560436d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:19.823200 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0a161-2b5c-4bfc-ad4a-5c128560436d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:19.824573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0a161-2b5c-4bfc-ad4a-5c128560436d None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:19.833213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-5eb0a161-2b5c-4bfc-ad4a-5c128560436d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 09:01:19.833213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:19.833213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:01:19.833213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:01:19.833213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:01:19.833213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:01:19.833213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:01:19.833213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:01:19.833213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 09:01:19.833213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:01:19.833213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:01:19.833213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:01:19.833213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:19.833213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:01:19.833213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:01:19.833213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:01:19.833213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:01:19.834262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:01:19.834262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:01:19.834262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:01:19.834262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:01:19.834262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:01:19.834262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:01:19.834262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:01:19.834262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:01:19.834262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:01:19.834262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:01:19.834262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:01:19.834262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:01:19.834262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:01:19.834262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:01:19.834262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:01:19.834262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:01:19.835335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:01:19.835335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 09:01:19.835335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:01:19.835335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:01:19.835335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:01:19.835335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:01:19.835335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:01:19.835335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:01:19.835335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:01:19.835335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:01:19.835335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:01:19.835335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:01:19.835335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 09:01:19.835335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:01:19.835335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 645/1298] 10.222.0.100 () {56 vars in 1122 bytes} [Tue Aug 2 09:01:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:01:19.841041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8e4ea-63db-49bd-8c86-206689e4e666 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:19.859640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:19.940873 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8e4ea-63db-49bd-8c86-206689e4e666 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:19.974174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b30e9-408d-4a48-aa7d-3aca255826b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:19.974792 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b30e9-408d-4a48-aa7d-3aca255826b6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:19.975078 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b30e9-408d-4a48-aa7d-3aca255826b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:19.975355 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b30e9-408d-4a48-aa7d-3aca255826b6 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:19.976028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b30e9-408d-4a48-aa7d-3aca255826b6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:19.976336 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b30e9-408d-4a48-aa7d-3aca255826b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:19.977735 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b30e9-408d-4a48-aa7d-3aca255826b6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:19.983474 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 654/1299] 10.222.0.100 () {56 vars in 1121 bytes} [Tue Aug 2 09:01:19 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:01:19.989355 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0a161-2b5c-4bfc-ad4a-5c128560436d None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:20.020639 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:20.087121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0a161-2b5c-4bfc-ad4a-5c128560436d None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:20.120439 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beff73b-063f-4ea8-8f65-7e664d52f214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:20.121044 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beff73b-063f-4ea8-8f65-7e664d52f214 None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:20.121433 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beff73b-063f-4ea8-8f65-7e664d52f214 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:20.121735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beff73b-063f-4ea8-8f65-7e664d52f214 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:20.122481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beff73b-063f-4ea8-8f65-7e664d52f214 None admin] RBAC: Adding query filter params (interface=public, service_id=bfe1390d195549b39ec4b649e69144a5, region_id=RegionOne) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:20.122795 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beff73b-063f-4ea8-8f65-7e664d52f214 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:20.124156 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beff73b-063f-4ea8-8f65-7e664d52f214 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:20.129025 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 646/1300] 10.222.0.100 () {56 vars in 1256 bytes} [Tue Aug 2 09:01:19 2022] GET /identity/v3/endpoints?service_id=bfe1390d195549b39ec4b649e69144a5&interface=public®ion_id=RegionOne => generated 195 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:01:20.134635 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b30e9-408d-4a48-aa7d-3aca255826b6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:20.215632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:20.233731 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b30e9-408d-4a48-aa7d-3aca255826b6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:20.267015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538b4e8-fa1f-4688-8c2b-b5289ec1163a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:20.267634 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538b4e8-fa1f-4688-8c2b-b5289ec1163a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:20.267920 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538b4e8-fa1f-4688-8c2b-b5289ec1163a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:20.268199 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538b4e8-fa1f-4688-8c2b-b5289ec1163a None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:20.268847 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538b4e8-fa1f-4688-8c2b-b5289ec1163a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:20.270236 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538b4e8-fa1f-4688-8c2b-b5289ec1163a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:20.276896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 655/1301] 10.222.0.100 () {56 vars in 1092 bytes} [Tue Aug 2 09:01:20 2022] GET /identity/v3/services => generated 2201 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:01:22.259992 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547bc3c-3ac1-4999-aa5f-81b10c5f5f79 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:22.260519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547bc3c-3ac1-4999-aa5f-81b10c5f5f79 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:22.260621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547bc3c-3ac1-4999-aa5f-81b10c5f5f79 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:22.262583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 647/1302] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:01:22 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:01:22.267904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb11118-f7aa-45d0-8ee6-4bb2bd04e8bd None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:22.268318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb11118-f7aa-45d0-8ee6-4bb2bd04e8bd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:22.268462 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb11118-f7aa-45d0-8ee6-4bb2bd04e8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:22.286634 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1bb11118-f7aa-45d0-8ee6-4bb2bd04e8bd None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:01:22.287721 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1bb11118-f7aa-45d0-8ee6-4bb2bd04e8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:01:22.352501 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 656/1303] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:01:22 2022] POST /identity/v3/auth/tokens => generated 2226 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:01:22.375427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547bc3c-3ac1-4999-aa5f-81b10c5f5f79 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:22.469441 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:22.476633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547bc3c-3ac1-4999-aa5f-81b10c5f5f79 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:22.511047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1776c79-6708-4611-a53b-c567b09ed6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:22.511617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1776c79-6708-4611-a53b-c567b09ed6ee None admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:22.511903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1776c79-6708-4611-a53b-c567b09ed6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:22.512182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1776c79-6708-4611-a53b-c567b09ed6ee None admin] PATH_INFO: `/v3/services/placement` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:22.512805 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1776c79-6708-4611-a53b-c567b09ed6ee None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:22.514211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1776c79-6708-4611-a53b-c567b09ed6ee None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:22.523022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application [None req-f1776c79-6708-4611-a53b-c567b09ed6ee None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 09:01:22.523022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:22.523022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:01:22.523022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:01:22.523022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:01:22.523022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:01:22.523022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:01:22.523022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:01:22.523022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 09:01:22.523022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:01:22.523022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:01:22.523022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:01:22.523022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:01:22.523022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:01:22.523022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:01:22.523022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:01:22.523022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:01:22.524224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:01:22.524224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:01:22.524224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:01:22.524224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:01:22.524224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:01:22.524224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:01:22.524224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:01:22.524224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:01:22.524224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:01:22.524224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:01:22.524224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:01:22.524224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:01:22.524224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:01:22.524224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:01:22.524224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:01:22.524224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:01:22.525282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:01:22.525282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application with Lock( Aug 02 09:01:22.525282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:01:22.525282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:01:22.525282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:01:22.525282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:01:22.525282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:01:22.525282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:01:22.525282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:01:22.525282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:01:22.525282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:01:22.525282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:01:22.525282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 09:01:22.525282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: ERROR keystone.server.flask.application Aug 02 09:01:22.525282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 648/1304] 10.222.0.100 () {56 vars in 1122 bytes} [Tue Aug 2 09:01:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:01:22.530375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb11118-f7aa-45d0-8ee6-4bb2bd04e8bd None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:22.617166 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:22.644517 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb11118-f7aa-45d0-8ee6-4bb2bd04e8bd None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:22.679934 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f669d53-fe1d-4881-8f7c-166555421f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:22.680550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f669d53-fe1d-4881-8f7c-166555421f9c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:22.680832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f669d53-fe1d-4881-8f7c-166555421f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:22.681131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f669d53-fe1d-4881-8f7c-166555421f9c None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:22.681804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f669d53-fe1d-4881-8f7c-166555421f9c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:01:22.682108 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f669d53-fe1d-4881-8f7c-166555421f9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:22.683456 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f669d53-fe1d-4881-8f7c-166555421f9c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:22.688908 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 657/1305] 10.222.0.100 () {56 vars in 1121 bytes} [Tue Aug 2 09:01:22 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:01:22.694715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1776c79-6708-4611-a53b-c567b09ed6ee None admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:01:22.792134 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1776c79-6708-4611-a53b-c567b09ed6ee None admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:01:22.825340 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfc452-43c4-49a7-a879-9148b9ccd104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:01:22.825946 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfc452-43c4-49a7-a879-9148b9ccd104 None admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:01:22.826229 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfc452-43c4-49a7-a879-9148b9ccd104 None admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:01:22.826503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfc452-43c4-49a7-a879-9148b9ccd104 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:01:22.827201 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfc452-43c4-49a7-a879-9148b9ccd104 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:01:22.828560 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfc452-43c4-49a7-a879-9148b9ccd104 None admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:01:22.847094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 649/1306] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:01:22 2022] POST /identity/v3/endpoints => generated 333 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:01:32.685974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=30 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:01:32.842920 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:25.797993 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897dd66e-b9e5-437f-9b88-df73ad06eddd None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:25.798580 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897dd66e-b9e5-437f-9b88-df73ad06eddd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:25.798580 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897dd66e-b9e5-437f-9b88-df73ad06eddd None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:25.800161 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 658/1307] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:02:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:02:25.806038 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a362b52-a673-405e-ad5f-67d0cc0a27eb None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:25.806468 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a362b52-a673-405e-ad5f-67d0cc0a27eb None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:25.806645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a362b52-a673-405e-ad5f-67d0cc0a27eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:25.886749 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-1a362b52-a673-405e-ad5f-67d0cc0a27eb None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:02:25.887967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1a362b52-a673-405e-ad5f-67d0cc0a27eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:26.009466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 650/1308] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:02:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:02:26.071453 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1cbe6-21ed-4dfa-88b3-1ee3f4cf0166 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:26.071913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1cbe6-21ed-4dfa-88b3-1ee3f4cf0166 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:26.072362 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1cbe6-21ed-4dfa-88b3-1ee3f4cf0166 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:26.074259 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 659/1309] 10.222.0.100 () {54 vars in 970 bytes} [Tue Aug 2 09:02:26 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:02:26.080090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca39c6d-cd5b-4891-b6c8-2580c7169db4 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:26.080398 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca39c6d-cd5b-4891-b6c8-2580c7169db4 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:26.081200 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca39c6d-cd5b-4891-b6c8-2580c7169db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:26.166734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-4ca39c6d-cd5b-4891-b6c8-2580c7169db4 None None] MFA Rules not processed for user `f4afef33deb247509765a8c88a78ac39`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:02:26.168066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-4ca39c6d-cd5b-4891-b6c8-2580c7169db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:26.193503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:26.269780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 651/1310] 10.222.0.100 () {58 vars in 1067 bytes} [Tue Aug 2 09:02:26 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:02:26.279149 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1cbe6-21ed-4dfa-88b3-1ee3f4cf0166 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:26.280998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5df1cbe6-21ed-4dfa-88b3-1ee3f4cf0166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:26.410981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1cbe6-21ed-4dfa-88b3-1ee3f4cf0166 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:26.489415 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ffb283-e2b4-4e46-8729-96fcfa695323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:26.490045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffb283-e2b4-4e46-8729-96fcfa695323 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:26.490330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffb283-e2b4-4e46-8729-96fcfa695323 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:26.490605 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffb283-e2b4-4e46-8729-96fcfa695323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:26.492759 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e8ffb283-e2b4-4e46-8729-96fcfa695323 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:26.550802 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:26.613090 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ffb283-e2b4-4e46-8729-96fcfa695323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:26.614603 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ffb283-e2b4-4e46-8729-96fcfa695323 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:26.638920 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 660/1311] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:02:26 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:02:28.929590 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e315086-d7ec-41dd-b931-dd5e6b933dc6 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:28.930083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e315086-d7ec-41dd-b931-dd5e6b933dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:28.930310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e315086-d7ec-41dd-b931-dd5e6b933dc6 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:28.931790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 652/1312] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:02:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:02:28.937192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155028b-f8e0-4187-a028-7dd250a6c55b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:28.937650 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155028b-f8e0-4187-a028-7dd250a6c55b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:28.937883 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155028b-f8e0-4187-a028-7dd250a6c55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:29.020670 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a155028b-f8e0-4187-a028-7dd250a6c55b None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:02:29.021832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a155028b-f8e0-4187-a028-7dd250a6c55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:29.134964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 661/1313] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:02:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:02:29.173173 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e315086-d7ec-41dd-b931-dd5e6b933dc6 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:29.174970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-9e315086-d7ec-41dd-b931-dd5e6b933dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:29.282718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e315086-d7ec-41dd-b931-dd5e6b933dc6 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:29.358963 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:29.361759 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1aa02-3386-4a17-80cd-1bf36b6eddf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:29.362441 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1aa02-3386-4a17-80cd-1bf36b6eddf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:29.362982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1aa02-3386-4a17-80cd-1bf36b6eddf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:29.363442 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1aa02-3386-4a17-80cd-1bf36b6eddf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:29.365505 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-63f1aa02-3386-4a17-80cd-1bf36b6eddf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:29.477865 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f1aa02-3386-4a17-80cd-1bf36b6eddf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:29.479248 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f1aa02-3386-4a17-80cd-1bf36b6eddf2 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:29.498392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 653/1314] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:02:29 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:02:31.821658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d7d19-53ce-4212-8d58-774bae2d5032 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:31.822259 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d7d19-53ce-4212-8d58-774bae2d5032 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:31.822259 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d7d19-53ce-4212-8d58-774bae2d5032 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:31.823733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 662/1315] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:02:31 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:02:31.828779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39baf6de-5bfc-4a05-b9af-31110823b196 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:31.829214 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39baf6de-5bfc-4a05-b9af-31110823b196 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:31.829384 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39baf6de-5bfc-4a05-b9af-31110823b196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:31.896444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-39baf6de-5bfc-4a05-b9af-31110823b196 None None] MFA Rules not processed for user `b44229d01ea54d08a04c913b5f878f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:02:31.897663 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-39baf6de-5bfc-4a05-b9af-31110823b196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:31.952872 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:31.998936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 654/1316] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:02:31 2022] POST /identity/v3/auth/tokens => generated 3253 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:02:32.039842 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d7d19-53ce-4212-8d58-774bae2d5032 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:32.041424 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8e4d7d19-53ce-4212-8d58-774bae2d5032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:32.051197 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:32.147358 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d7d19-53ce-4212-8d58-774bae2d5032 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:32.220316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55644ce-aea2-4573-b28a-1e35db668518 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:32.220940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55644ce-aea2-4573-b28a-1e35db668518 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:32.221259 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55644ce-aea2-4573-b28a-1e35db668518 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:32.221536 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55644ce-aea2-4573-b28a-1e35db668518 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:32.223159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a55644ce-aea2-4573-b28a-1e35db668518 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:32.285682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:32.332270 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55644ce-aea2-4573-b28a-1e35db668518 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:32.333739 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55644ce-aea2-4573-b28a-1e35db668518 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:32.351794 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 663/1317] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:02:32 2022] GET /identity/v3/auth/tokens => generated 3253 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:02:34.942315 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4802341-c468-4f80-bfae-b605ee19c58a None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:34.942801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4802341-c468-4f80-bfae-b605ee19c58a None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:34.942896 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4802341-c468-4f80-bfae-b605ee19c58a None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:34.944228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 655/1318] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:02:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:02:34.949858 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8af014-89b3-459f-9c50-6937f322b996 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:34.950254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8af014-89b3-459f-9c50-6937f322b996 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:34.950423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8af014-89b3-459f-9c50-6937f322b996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:35.015462 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-0c8af014-89b3-459f-9c50-6937f322b996 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:02:35.016666 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0c8af014-89b3-459f-9c50-6937f322b996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:35.112788 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 664/1319] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:02:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:02:35.151988 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4802341-c468-4f80-bfae-b605ee19c58a None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:35.153606 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-a4802341-c468-4f80-bfae-b605ee19c58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:35.255420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4802341-c468-4f80-bfae-b605ee19c58a None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:35.328465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b827a36-a561-4147-812d-a56e0e6a7018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:35.329068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b827a36-a561-4147-812d-a56e0e6a7018 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:35.329377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b827a36-a561-4147-812d-a56e0e6a7018 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:35.329647 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b827a36-a561-4147-812d-a56e0e6a7018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:35.331224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-6b827a36-a561-4147-812d-a56e0e6a7018 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:35.338194 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:35.436493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b827a36-a561-4147-812d-a56e0e6a7018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:35.437955 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b827a36-a561-4147-812d-a56e0e6a7018 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:35.456316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 656/1320] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:02:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:02:38.132388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1869c00-2ce1-42f6-8b23-2217b9cab647 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:38.132960 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1869c00-2ce1-42f6-8b23-2217b9cab647 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:38.133073 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1869c00-2ce1-42f6-8b23-2217b9cab647 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:38.134618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 665/1321] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:02:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:02:38.139868 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb025e-325b-460d-999e-be37c3405941 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:38.140250 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb025e-325b-460d-999e-be37c3405941 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:38.140435 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb025e-325b-460d-999e-be37c3405941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:38.206931 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-edfb025e-325b-460d-999e-be37c3405941 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:02:38.208050 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-edfb025e-325b-460d-999e-be37c3405941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:38.275024 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:38.318861 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 657/1322] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:02:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:02:38.355832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1869c00-2ce1-42f6-8b23-2217b9cab647 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:38.357396 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c1869c00-2ce1-42f6-8b23-2217b9cab647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:38.378326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:38.469708 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1869c00-2ce1-42f6-8b23-2217b9cab647 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:38.544971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13371629-48f8-4a87-8950-5f4e13931123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:38.545633 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13371629-48f8-4a87-8950-5f4e13931123 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:38.545918 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13371629-48f8-4a87-8950-5f4e13931123 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:38.546194 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13371629-48f8-4a87-8950-5f4e13931123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:38.547803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-13371629-48f8-4a87-8950-5f4e13931123 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:38.621976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:38.660912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13371629-48f8-4a87-8950-5f4e13931123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:38.662364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13371629-48f8-4a87-8950-5f4e13931123 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:38.680529 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 666/1323] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:02:38 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:02:40.744287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145675f3-ddb5-4c65-85be-583951757f09 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:40.744855 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145675f3-ddb5-4c65-85be-583951757f09 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:40.744855 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145675f3-ddb5-4c65-85be-583951757f09 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:40.746208 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 658/1324] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:02:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:02:40.751298 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd415135-d1e1-451c-8050-5002f2cb9727 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:40.751743 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd415135-d1e1-451c-8050-5002f2cb9727 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:40.751869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd415135-d1e1-451c-8050-5002f2cb9727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:40.818575 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-dd415135-d1e1-451c-8050-5002f2cb9727 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:02:40.820248 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-dd415135-d1e1-451c-8050-5002f2cb9727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:40.921236 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 667/1325] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:02:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:02:40.956774 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145675f3-ddb5-4c65-85be-583951757f09 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:40.958416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-145675f3-ddb5-4c65-85be-583951757f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:41.062413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145675f3-ddb5-4c65-85be-583951757f09 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:41.136394 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3be507-82d9-459b-b12a-5a69bef3060b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:41.137017 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3be507-82d9-459b-b12a-5a69bef3060b service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:41.137332 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3be507-82d9-459b-b12a-5a69bef3060b service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:41.137608 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3be507-82d9-459b-b12a-5a69bef3060b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:41.139266 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-bc3be507-82d9-459b-b12a-5a69bef3060b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:41.155929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:41.253956 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3be507-82d9-459b-b12a-5a69bef3060b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:41.255402 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3be507-82d9-459b-b12a-5a69bef3060b service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:41.275026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 659/1326] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:02:40 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:02:43.339910 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36257a-a5d3-45ec-b802-6dd57e5606ca None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:43.340443 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36257a-a5d3-45ec-b802-6dd57e5606ca None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:43.340443 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36257a-a5d3-45ec-b802-6dd57e5606ca None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:43.341868 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 668/1327] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:02:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:02:43.346965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4724e2-5c04-4e57-a9fa-c48198e195f7 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:43.347364 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4724e2-5c04-4e57-a9fa-c48198e195f7 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:43.347518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4724e2-5c04-4e57-a9fa-c48198e195f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:43.416783 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-5d4724e2-5c04-4e57-a9fa-c48198e195f7 None None] MFA Rules not processed for user `b44229d01ea54d08a04c913b5f878f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:02:43.418019 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-5d4724e2-5c04-4e57-a9fa-c48198e195f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:43.484295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:43.519511 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 660/1328] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:02:43 2022] POST /identity/v3/auth/tokens => generated 3253 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:02:43.556974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36257a-a5d3-45ec-b802-6dd57e5606ca None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:43.558510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6d36257a-a5d3-45ec-b802-6dd57e5606ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:43.592049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:43.671593 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36257a-a5d3-45ec-b802-6dd57e5606ca None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:43.745210 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f81340-d421-4bcc-af3d-b4a3406e349c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:43.745836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f81340-d421-4bcc-af3d-b4a3406e349c service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:43.746122 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f81340-d421-4bcc-af3d-b4a3406e349c service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:43.746398 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f81340-d421-4bcc-af3d-b4a3406e349c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:43.748017 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e3f81340-d421-4bcc-af3d-b4a3406e349c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:43.828192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:43.861858 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f81340-d421-4bcc-af3d-b4a3406e349c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:43.863285 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f81340-d421-4bcc-af3d-b4a3406e349c service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:43.881351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 669/1329] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:02:43 2022] GET /identity/v3/auth/tokens => generated 3253 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:02:46.161555 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0bf6f-5606-4afd-8e2e-83cc46cc537d None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:46.162058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0bf6f-5606-4afd-8e2e-83cc46cc537d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:46.162058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0bf6f-5606-4afd-8e2e-83cc46cc537d None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:46.163803 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 661/1330] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:02:46 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:02:46.168937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8450c4-8436-4f9e-96d1-6932cb453f29 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:46.169352 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8450c4-8436-4f9e-96d1-6932cb453f29 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:46.169556 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8450c4-8436-4f9e-96d1-6932cb453f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:46.235131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9f8450c4-8436-4f9e-96d1-6932cb453f29 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:02:46.236336 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9f8450c4-8436-4f9e-96d1-6932cb453f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:46.332786 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 670/1331] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:02:46 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:02:46.383431 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c861d2b-e982-4276-bdd3-74938b393e9c None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:46.383716 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c861d2b-e982-4276-bdd3-74938b393e9c None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:46.383998 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c861d2b-e982-4276-bdd3-74938b393e9c None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:46.385425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 662/1332] 10.222.0.100 () {54 vars in 970 bytes} [Tue Aug 2 09:02:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:02:46.391026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee132a7-1322-4e34-b064-150942bb62b6 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:46.391315 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee132a7-1322-4e34-b064-150942bb62b6 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:46.391589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee132a7-1322-4e34-b064-150942bb62b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:46.437596 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:46.469038 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-fee132a7-1322-4e34-b064-150942bb62b6 None None] MFA Rules not processed for user `f4afef33deb247509765a8c88a78ac39`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:02:46.470272 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fee132a7-1322-4e34-b064-150942bb62b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:46.547456 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 671/1333] 10.222.0.100 () {58 vars in 1067 bytes} [Tue Aug 2 09:02:46 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:02:46.556863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c861d2b-e982-4276-bdd3-74938b393e9c None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:46.558417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-0c861d2b-e982-4276-bdd3-74938b393e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:46.661684 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c861d2b-e982-4276-bdd3-74938b393e9c None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:46.734619 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092bf14-281e-4aa6-8a89-e29067ac6360 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:46.735163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092bf14-281e-4aa6-8a89-e29067ac6360 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:46.735441 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092bf14-281e-4aa6-8a89-e29067ac6360 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:46.735714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092bf14-281e-4aa6-8a89-e29067ac6360 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:46.737346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-2092bf14-281e-4aa6-8a89-e29067ac6360 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:46.765282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:46.848121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092bf14-281e-4aa6-8a89-e29067ac6360 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:46.849601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092bf14-281e-4aa6-8a89-e29067ac6360 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:46.868802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 663/1334] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:02:46 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:02:49.124375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e868c-1e23-4d0b-ba32-161dbafa8bd0 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:49.124904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e868c-1e23-4d0b-ba32-161dbafa8bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:49.125001 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e868c-1e23-4d0b-ba32-161dbafa8bd0 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:49.126364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 672/1335] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:02:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:02:49.131498 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596e824-0023-4423-82a9-25f630af4203 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:49.131884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596e824-0023-4423-82a9-25f630af4203 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:49.132041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596e824-0023-4423-82a9-25f630af4203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:49.195923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-5596e824-0023-4423-82a9-25f630af4203 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:02:49.197052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-5596e824-0023-4423-82a9-25f630af4203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:49.268883 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:49.296767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 664/1336] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:02:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:02:49.336871 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e868c-1e23-4d0b-ba32-161dbafa8bd0 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:49.338441 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ea9e868c-1e23-4d0b-ba32-161dbafa8bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:49.441157 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e868c-1e23-4d0b-ba32-161dbafa8bd0 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:49.469483 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:49.516256 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95063467-aee0-483f-8953-b88491fa5b59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:49.516927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95063467-aee0-483f-8953-b88491fa5b59 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:49.517251 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95063467-aee0-483f-8953-b88491fa5b59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:49.517531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95063467-aee0-483f-8953-b88491fa5b59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:49.519142 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-95063467-aee0-483f-8953-b88491fa5b59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:49.626689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95063467-aee0-483f-8953-b88491fa5b59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:49.628118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95063467-aee0-483f-8953-b88491fa5b59 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:49.645946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 673/1337] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:02:49 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:02:52.311275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b2e10-5a5c-4f72-a42d-50ab34b16e66 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:52.311853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b2e10-5a5c-4f72-a42d-50ab34b16e66 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:52.311853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b2e10-5a5c-4f72-a42d-50ab34b16e66 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:52.313228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 665/1338] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:02:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:02:52.318442 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802a4da-1d40-4f8b-a0fd-48a65435e1c7 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:52.318986 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802a4da-1d40-4f8b-a0fd-48a65435e1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:52.318986 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802a4da-1d40-4f8b-a0fd-48a65435e1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:52.384726 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-7802a4da-1d40-4f8b-a0fd-48a65435e1c7 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:02:52.385964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7802a4da-1d40-4f8b-a0fd-48a65435e1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:52.394112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:52.485142 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 674/1339] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:02:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:02:52.523212 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b2e10-5a5c-4f72-a42d-50ab34b16e66 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:52.524795 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1a3b2e10-5a5c-4f72-a42d-50ab34b16e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:52.636407 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b2e10-5a5c-4f72-a42d-50ab34b16e66 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:52.708395 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5160c-ae9a-4f67-b207-b5e6340df42c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:52.709039 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5160c-ae9a-4f67-b207-b5e6340df42c service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:52.709351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5160c-ae9a-4f67-b207-b5e6340df42c service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:52.709710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5160c-ae9a-4f67-b207-b5e6340df42c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:52.711347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-eef5160c-ae9a-4f67-b207-b5e6340df42c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:52.746435 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:52.821825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5160c-ae9a-4f67-b207-b5e6340df42c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:52.823277 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5160c-ae9a-4f67-b207-b5e6340df42c service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:52.842559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 666/1340] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:02:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:02:56.917886 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703af5f-c5d3-498b-9e65-9bcf78f57648 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:56.918462 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703af5f-c5d3-498b-9e65-9bcf78f57648 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:56.918462 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703af5f-c5d3-498b-9e65-9bcf78f57648 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:56.920196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 675/1341] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:02:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:02:56.925055 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa6af2-002f-4681-8723-44d631b774fe None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:56.925554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa6af2-002f-4681-8723-44d631b774fe None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:56.925832 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa6af2-002f-4681-8723-44d631b774fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:56.992180 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-f2fa6af2-002f-4681-8723-44d631b774fe None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:02:56.993650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f2fa6af2-002f-4681-8723-44d631b774fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:57.080053 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:57.092719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 667/1342] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:02:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:02:57.132497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703af5f-c5d3-498b-9e65-9bcf78f57648 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:57.134147 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f703af5f-c5d3-498b-9e65-9bcf78f57648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:57.239283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703af5f-c5d3-498b-9e65-9bcf78f57648 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:57.279027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:57.315919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e3efe9-22ac-4145-b150-96dbb2f390d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:57.316544 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3efe9-22ac-4145-b150-96dbb2f390d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:57.316830 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3efe9-22ac-4145-b150-96dbb2f390d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:57.317121 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3efe9-22ac-4145-b150-96dbb2f390d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:57.318783 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f2e3efe9-22ac-4145-b150-96dbb2f390d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:57.429007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e3efe9-22ac-4145-b150-96dbb2f390d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:57.430520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e3efe9-22ac-4145-b150-96dbb2f390d0 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:57.452217 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 676/1343] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:02:57 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:03:00.091194 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f810d9-581e-43d8-9f1e-af85ffd0e372 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:00.091737 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f810d9-581e-43d8-9f1e-af85ffd0e372 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:00.091737 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f810d9-581e-43d8-9f1e-af85ffd0e372 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:00.093173 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 668/1344] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:03:00 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:00.098715 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d4786-4b6d-49ad-a1d9-78cb4d96c1dc None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:00.099342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d4786-4b6d-49ad-a1d9-78cb4d96c1dc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:00.099342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d4786-4b6d-49ad-a1d9-78cb4d96c1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:00.169983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ca4d4786-4b6d-49ad-a1d9-78cb4d96c1dc None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:00.171111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ca4d4786-4b6d-49ad-a1d9-78cb4d96c1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:00.189841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:00.271690 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 677/1345] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:03:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:03:00.309569 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f810d9-581e-43d8-9f1e-af85ffd0e372 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:03:00.311090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-83f810d9-581e-43d8-9f1e-af85ffd0e372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:00.414908 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f810d9-581e-43d8-9f1e-af85ffd0e372 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:03:00.489443 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88e33c-a22c-4f33-bc1a-d3db7feb24e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:03:00.490607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88e33c-a22c-4f33-bc1a-d3db7feb24e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:00.490893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88e33c-a22c-4f33-bc1a-d3db7feb24e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:00.491163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88e33c-a22c-4f33-bc1a-d3db7feb24e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:00.493035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-2c88e33c-a22c-4f33-bc1a-d3db7feb24e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:00.536133 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:00.609985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88e33c-a22c-4f33-bc1a-d3db7feb24e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:03:00.611423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88e33c-a22c-4f33-bc1a-d3db7feb24e4 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:03:00.631200 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 669/1346] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:03:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:03:04.628133 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e421e4-c23f-4dc7-bee0-6582e25c3157 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:04.628665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e421e4-c23f-4dc7-bee0-6582e25c3157 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:04.628665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e421e4-c23f-4dc7-bee0-6582e25c3157 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:04.631027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 678/1347] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:03:04 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:04.636281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e06799-c62f-41d0-9f2c-2cc7e7a2bdae None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:04.636759 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e06799-c62f-41d0-9f2c-2cc7e7a2bdae None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:04.636876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e06799-c62f-41d0-9f2c-2cc7e7a2bdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:04.703593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-e8e06799-c62f-41d0-9f2c-2cc7e7a2bdae None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:04.704707 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e8e06799-c62f-41d0-9f2c-2cc7e7a2bdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:04.802518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:04.805582 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 670/1348] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:03:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:03:04.844070 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e421e4-c23f-4dc7-bee0-6582e25c3157 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:03:04.846127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-87e421e4-c23f-4dc7-bee0-6582e25c3157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:04.983325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e421e4-c23f-4dc7-bee0-6582e25c3157 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:03:05.038998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:05.067811 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54513e2f-c770-4d11-a414-fa3ec94d55c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:03:05.068366 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54513e2f-c770-4d11-a414-fa3ec94d55c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:05.068650 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54513e2f-c770-4d11-a414-fa3ec94d55c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:05.068920 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54513e2f-c770-4d11-a414-fa3ec94d55c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:05.070604 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-54513e2f-c770-4d11-a414-fa3ec94d55c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:05.188623 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54513e2f-c770-4d11-a414-fa3ec94d55c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:03:05.190090 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54513e2f-c770-4d11-a414-fa3ec94d55c1 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:03:05.210603 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 679/1349] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:03:04 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:03:10.340651 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3bbdf-553b-4774-94c6-699869842760 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:10.341142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3bbdf-553b-4774-94c6-699869842760 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:10.341263 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3bbdf-553b-4774-94c6-699869842760 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:10.342921 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 671/1350] 10.222.0.100 () {54 vars in 926 bytes} [Tue Aug 2 09:03:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:10.347977 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108bcf4-0b9d-4380-b660-6ca68b844c15 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:10.348938 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108bcf4-0b9d-4380-b660-6ca68b844c15 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:10.349138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108bcf4-0b9d-4380-b660-6ca68b844c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:10.523234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-2108bcf4-0b9d-4380-b660-6ca68b844c15 None None] MFA Rules not processed for user `b40981fe179a43dc81553d3594dad649`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:10.524658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2108bcf4-0b9d-4380-b660-6ca68b844c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:10.547929 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:10.673453 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 680/1351] 10.222.0.100 () {58 vars in 1023 bytes} [Tue Aug 2 09:03:10 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:03:15.134449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a2715-f179-4e83-a8d7-3da41f24a15b None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:15.134972 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a2715-f179-4e83-a8d7-3da41f24a15b None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:15.134972 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a2715-f179-4e83-a8d7-3da41f24a15b None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:15.136555 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 672/1352] 10.222.0.100 () {54 vars in 926 bytes} [Tue Aug 2 09:03:15 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:15.141835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888100f-62dc-45ff-9212-69b6cf754101 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:15.142666 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888100f-62dc-45ff-9212-69b6cf754101 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:15.143252 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888100f-62dc-45ff-9212-69b6cf754101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:15.233401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d888100f-62dc-45ff-9212-69b6cf754101 None None] MFA Rules not processed for user `b40981fe179a43dc81553d3594dad649`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:15.234974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d888100f-62dc-45ff-9212-69b6cf754101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:15.328200 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:15.372246 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 681/1353] 10.222.0.100 () {58 vars in 1023 bytes} [Tue Aug 2 09:03:15 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:03:17.077158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6fe39-095f-4e5b-bcfd-af19a8391536 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:17.077766 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6fe39-095f-4e5b-bcfd-af19a8391536 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:17.077766 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6fe39-095f-4e5b-bcfd-af19a8391536 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:17.079066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 673/1354] 10.222.0.100 () {54 vars in 926 bytes} [Tue Aug 2 09:03:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:17.084754 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086adb8-e80f-41c0-9012-fe1d9a432f05 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:17.085229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086adb8-e80f-41c0-9012-fe1d9a432f05 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:17.085326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086adb8-e80f-41c0-9012-fe1d9a432f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:17.169365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6086adb8-e80f-41c0-9012-fe1d9a432f05 None None] MFA Rules not processed for user `b40981fe179a43dc81553d3594dad649`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:17.170640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6086adb8-e80f-41c0-9012-fe1d9a432f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:17.285711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 682/1355] 10.222.0.100 () {58 vars in 1023 bytes} [Tue Aug 2 09:03:17 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:03:20.878354 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf1213-d854-4955-acf4-049b73ddf59b None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:20.879655 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf1213-d854-4955-acf4-049b73ddf59b None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:20.881144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf1213-d854-4955-acf4-049b73ddf59b None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:20.887140 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 674/1356] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:03:20 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:20.889446 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59609e1-e3a2-48a4-914a-2583f4220291 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:20.889908 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59609e1-e3a2-48a4-914a-2583f4220291 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:20.889999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59609e1-e3a2-48a4-914a-2583f4220291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:20.987425 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:21.033588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a59609e1-e3a2-48a4-914a-2583f4220291 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:21.034943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a59609e1-e3a2-48a4-914a-2583f4220291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:21.150745 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 683/1357] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:03:20 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:03:21.208918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93420fdb-7e02-4a7c-a91f-d63bb0c8477c None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:21.209526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93420fdb-7e02-4a7c-a91f-d63bb0c8477c None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:21.210024 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93420fdb-7e02-4a7c-a91f-d63bb0c8477c None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:21.211877 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 675/1358] 10.222.0.100 () {54 vars in 967 bytes} [Tue Aug 2 09:03:21 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:21.219239 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743147b-64c7-41e9-8b40-554b39cbcc91 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:21.219532 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743147b-64c7-41e9-8b40-554b39cbcc91 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:21.219810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743147b-64c7-41e9-8b40-554b39cbcc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:21.303873 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e743147b-64c7-41e9-8b40-554b39cbcc91 None None] MFA Rules not processed for user `49a0c1a4a6cf43e28ec1b8488c681848`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:21.305090 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e743147b-64c7-41e9-8b40-554b39cbcc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:21.362661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:21.443010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 684/1359] 10.222.0.100 () {58 vars in 1064 bytes} [Tue Aug 2 09:03:21 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:03:21.450773 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93420fdb-7e02-4a7c-a91f-d63bb0c8477c None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:03:21.452284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-93420fdb-7e02-4a7c-a91f-d63bb0c8477c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:21.607820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93420fdb-7e02-4a7c-a91f-d63bb0c8477c None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:03:21.686051 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:21.732227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca5f01-169b-4d32-817e-d08c1e783ec5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:03:21.732917 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca5f01-169b-4d32-817e-d08c1e783ec5 service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:21.733196 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca5f01-169b-4d32-817e-d08c1e783ec5 service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:21.733472 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca5f01-169b-4d32-817e-d08c1e783ec5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:21.735055 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e6ca5f01-169b-4d32-817e-d08c1e783ec5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:21.835195 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca5f01-169b-4d32-817e-d08c1e783ec5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:03:21.836796 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca5f01-169b-4d32-817e-d08c1e783ec5 service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:03:21.857500 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 676/1360] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:03:21 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:03:24.319331 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cd792-2cd8-48bd-879e-eb4a570a2cc3 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:24.319835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cd792-2cd8-48bd-879e-eb4a570a2cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:24.319949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cd792-2cd8-48bd-879e-eb4a570a2cc3 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:24.321470 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 685/1361] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:03:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:24.326817 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6172537-fef2-4943-a53f-32839046889b None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:24.327495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6172537-fef2-4943-a53f-32839046889b None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:24.327495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6172537-fef2-4943-a53f-32839046889b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:24.398865 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-d6172537-fef2-4943-a53f-32839046889b None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:24.399815 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d6172537-fef2-4943-a53f-32839046889b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:24.455045 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:24.535621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 677/1362] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:03:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:03:24.581052 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cd792-2cd8-48bd-879e-eb4a570a2cc3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:03:24.584279 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-024cd792-2cd8-48bd-879e-eb4a570a2cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:24.811824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cd792-2cd8-48bd-879e-eb4a570a2cc3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:03:24.823250 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:24.995084 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b021840-12af-4acd-a2ed-35a86762c414 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:03:24.995084 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b021840-12af-4acd-a2ed-35a86762c414 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:24.995084 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b021840-12af-4acd-a2ed-35a86762c414 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:24.995084 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b021840-12af-4acd-a2ed-35a86762c414 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:24.996664 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2b021840-12af-4acd-a2ed-35a86762c414 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:25.107745 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b021840-12af-4acd-a2ed-35a86762c414 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:03:25.109915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b021840-12af-4acd-a2ed-35a86762c414 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:03:25.133675 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 686/1363] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:03:24 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:03:27.509401 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a852668-b283-4571-9d2b-fb0440865373 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:27.509917 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a852668-b283-4571-9d2b-fb0440865373 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:27.510021 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a852668-b283-4571-9d2b-fb0440865373 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:27.511317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 678/1364] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:03:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:27.517296 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb0654-424e-4916-a14c-efeda4f9c2b4 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:27.517757 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb0654-424e-4916-a14c-efeda4f9c2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:27.517757 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb0654-424e-4916-a14c-efeda4f9c2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:27.535947 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:27.605091 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8beb0654-424e-4916-a14c-efeda4f9c2b4 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:27.606594 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8beb0654-424e-4916-a14c-efeda4f9c2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:27.731407 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 687/1365] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:03:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:03:27.778484 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3015a-f9c5-4401-87b9-81e96e4ef996 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:27.778789 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3015a-f9c5-4401-87b9-81e96e4ef996 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:27.779053 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3015a-f9c5-4401-87b9-81e96e4ef996 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:27.780998 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 679/1366] 10.222.0.100 () {54 vars in 967 bytes} [Tue Aug 2 09:03:27 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:27.786559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a495352-a3e8-4a58-954d-0e706551c8d9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:27.786887 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a495352-a3e8-4a58-954d-0e706551c8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:27.787389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a495352-a3e8-4a58-954d-0e706551c8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:27.877417 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1a495352-a3e8-4a58-954d-0e706551c8d9 None None] MFA Rules not processed for user `49a0c1a4a6cf43e28ec1b8488c681848`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:27.878858 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1a495352-a3e8-4a58-954d-0e706551c8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:27.924937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:28.017013 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 688/1367] 10.222.0.100 () {58 vars in 1064 bytes} [Tue Aug 2 09:03:27 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:03:28.027972 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b3015a-f9c5-4401-87b9-81e96e4ef996 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:03:28.029640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f3b3015a-f9c5-4401-87b9-81e96e4ef996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:28.188342 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b3015a-f9c5-4401-87b9-81e96e4ef996 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:03:28.211877 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:28.316410 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f1251-3973-4976-861e-fa6848a2d5b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:03:28.316943 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f1251-3973-4976-861e-fa6848a2d5b0 service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:28.317257 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f1251-3973-4976-861e-fa6848a2d5b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:28.317534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f1251-3973-4976-861e-fa6848a2d5b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:28.319135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c72f1251-3973-4976-861e-fa6848a2d5b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:28.415314 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f1251-3973-4976-861e-fa6848a2d5b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:03:28.416914 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f1251-3973-4976-861e-fa6848a2d5b0 service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:03:28.436496 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 680/1368] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:03:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:03:29.577394 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d19a44-bdad-4cad-8f97-80c0228e7324 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:29.577942 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d19a44-bdad-4cad-8f97-80c0228e7324 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:29.577942 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d19a44-bdad-4cad-8f97-80c0228e7324 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:29.579493 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 689/1369] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:03:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:29.585770 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33270d6c-91a9-4009-8ee2-1c763b941b27 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:29.585770 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33270d6c-91a9-4009-8ee2-1c763b941b27 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:29.586357 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33270d6c-91a9-4009-8ee2-1c763b941b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:29.621810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:29.663426 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-33270d6c-91a9-4009-8ee2-1c763b941b27 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:29.664637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-33270d6c-91a9-4009-8ee2-1c763b941b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:29.775081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 681/1370] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:03:29 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:03:30.151123 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99cd38-5a6e-4ce1-b1ff-e640132d5cbb None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:30.151123 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99cd38-5a6e-4ce1-b1ff-e640132d5cbb None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:30.151123 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99cd38-5a6e-4ce1-b1ff-e640132d5cbb None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:30.151123 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 690/1371] 10.222.0.100 () {54 vars in 969 bytes} [Tue Aug 2 09:03:30 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:30.156522 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c2c7c-3904-4ddb-880b-6c2a5b37d2e1 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:30.157034 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c2c7c-3904-4ddb-880b-6c2a5b37d2e1 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:30.157471 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c2c7c-3904-4ddb-880b-6c2a5b37d2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:30.246857 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-195c2c7c-3904-4ddb-880b-6c2a5b37d2e1 None None] MFA Rules not processed for user `8e55bb6b624c45a3a08306403570177d`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:30.248080 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-195c2c7c-3904-4ddb-880b-6c2a5b37d2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:30.303499 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:30.337916 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 682/1372] 10.222.0.100 () {58 vars in 1066 bytes} [Tue Aug 2 09:03:30 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:03:30.345408 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99cd38-5a6e-4ce1-b1ff-e640132d5cbb None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:03:30.346950 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ff99cd38-5a6e-4ce1-b1ff-e640132d5cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:30.466296 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99cd38-5a6e-4ce1-b1ff-e640132d5cbb None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:03:30.513765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:30.532868 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6999af-69f6-4528-be0a-9da41621f4e6 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:30.533631 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6999af-69f6-4528-be0a-9da41621f4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:30.534121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6999af-69f6-4528-be0a-9da41621f4e6 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:30.538261 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 683/1373] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:03:30 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:30.543199 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7dc14-c070-42fa-a7a5-0381a7112073 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:30.543831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7dc14-c070-42fa-a7a5-0381a7112073 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:30.544333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7dc14-c070-42fa-a7a5-0381a7112073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:30.553554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ffdcf-10b0-498b-8dbf-331934eacd2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:03:30.554184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ffdcf-10b0-498b-8dbf-331934eacd2e service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:30.554469 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ffdcf-10b0-498b-8dbf-331934eacd2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:30.554745 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ffdcf-10b0-498b-8dbf-331934eacd2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:30.556407 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8a0ffdcf-10b0-498b-8dbf-331934eacd2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:30.630660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-58b7dc14-c070-42fa-a7a5-0381a7112073 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:30.631973 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-58b7dc14-c070-42fa-a7a5-0381a7112073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:30.683683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ffdcf-10b0-498b-8dbf-331934eacd2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:03:30.685008 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ffdcf-10b0-498b-8dbf-331934eacd2e service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:03:30.707841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 691/1374] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:03:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:03:30.751075 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 684/1375] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:03:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:03:30.795339 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ffdcf-10b0-498b-8dbf-331934eacd2e service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:03:30.796657 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8a0ffdcf-10b0-498b-8dbf-331934eacd2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:30.874824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:30.915676 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ffdcf-10b0-498b-8dbf-331934eacd2e service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:03:31.074609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0818e-230a-46fb-8b85-4287674eb8c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:03:31.075803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0818e-230a-46fb-8b85-4287674eb8c3 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:31.076281 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0818e-230a-46fb-8b85-4287674eb8c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:31.076560 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0818e-230a-46fb-8b85-4287674eb8c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:31.078221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c7e0818e-230a-46fb-8b85-4287674eb8c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:31.162569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:31.174281 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0818e-230a-46fb-8b85-4287674eb8c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:03:31.175895 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0818e-230a-46fb-8b85-4287674eb8c3 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:03:31.194777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 692/1376] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:03:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:03:33.295061 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48051599-fe35-4e9c-be9e-bb6e27142786 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:33.295613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48051599-fe35-4e9c-be9e-bb6e27142786 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:33.295613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48051599-fe35-4e9c-be9e-bb6e27142786 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:33.297650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 685/1377] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:03:33 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:33.304150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97e890-b411-436e-b41f-d818294235af None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:33.304629 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97e890-b411-436e-b41f-d818294235af None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:33.304737 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97e890-b411-436e-b41f-d818294235af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:33.377092 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9c97e890-b411-436e-b41f-d818294235af None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:33.378583 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9c97e890-b411-436e-b41f-d818294235af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:33.479242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 693/1378] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:03:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:03:33.523955 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48051599-fe35-4e9c-be9e-bb6e27142786 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:03:33.525634 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-48051599-fe35-4e9c-be9e-bb6e27142786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:33.582903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:33.691935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48051599-fe35-4e9c-be9e-bb6e27142786 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:03:33.829963 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de722b4-abb4-4e35-92d5-b34feec243fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:03:33.830505 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de722b4-abb4-4e35-92d5-b34feec243fb service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:33.830789 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de722b4-abb4-4e35-92d5-b34feec243fb service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:33.831054 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de722b4-abb4-4e35-92d5-b34feec243fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:33.833364 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-0de722b4-abb4-4e35-92d5-b34feec243fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:33.845016 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:33.927656 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de722b4-abb4-4e35-92d5-b34feec243fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:03:33.929306 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de722b4-abb4-4e35-92d5-b34feec243fb service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:03:33.948989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 686/1379] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:03:33 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:03:36.181342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c9c7d-1fe3-4ae1-9219-a3051aa1211c None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:36.181880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c9c7d-1fe3-4ae1-9219-a3051aa1211c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:36.181880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c9c7d-1fe3-4ae1-9219-a3051aa1211c None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:36.183398 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 694/1380] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:03:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:36.188746 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125aba59-6ccb-46f6-ae88-f5d4eda8f8f7 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:36.189266 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125aba59-6ccb-46f6-ae88-f5d4eda8f8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:36.189348 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125aba59-6ccb-46f6-ae88-f5d4eda8f8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:36.271725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-125aba59-6ccb-46f6-ae88-f5d4eda8f8f7 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:36.272939 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-125aba59-6ccb-46f6-ae88-f5d4eda8f8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:36.375030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:36.392314 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 687/1381] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:03:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:03:36.438249 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c9c7d-1fe3-4ae1-9219-a3051aa1211c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:03:36.439660 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3f6c9c7d-1fe3-4ae1-9219-a3051aa1211c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:36.512695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:36.619274 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c9c7d-1fe3-4ae1-9219-a3051aa1211c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:03:36.747250 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039524ec-9a5e-497c-970b-24d850ca5ed5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:03:36.747874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039524ec-9a5e-497c-970b-24d850ca5ed5 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:36.748172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039524ec-9a5e-497c-970b-24d850ca5ed5 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:36.748437 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039524ec-9a5e-497c-970b-24d850ca5ed5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:36.750104 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-039524ec-9a5e-497c-970b-24d850ca5ed5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:36.774817 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:36.850108 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039524ec-9a5e-497c-970b-24d850ca5ed5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:03:36.851699 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039524ec-9a5e-497c-970b-24d850ca5ed5 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:03:36.871945 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 695/1382] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:03:36 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:03:38.996849 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d7775-c12b-4067-98fc-6e7e6fc342f7 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:38.997443 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d7775-c12b-4067-98fc-6e7e6fc342f7 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:38.997443 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d7775-c12b-4067-98fc-6e7e6fc342f7 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:38.998836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 688/1383] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:03:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 09:03:39.005387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe769b4-ad06-46c4-a8a3-12b8310c9986 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:39.005872 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe769b4-ad06-46c4-a8a3-12b8310c9986 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:39.005986 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe769b4-ad06-46c4-a8a3-12b8310c9986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:39.082600 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-cbe769b4-ad06-46c4-a8a3-12b8310c9986 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:39.084278 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-cbe769b4-ad06-46c4-a8a3-12b8310c9986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:39.208436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:39.212255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 696/1384] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:03:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:03:39.258085 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d7775-c12b-4067-98fc-6e7e6fc342f7 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:03:39.259610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-a99d7775-c12b-4067-98fc-6e7e6fc342f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:39.446503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d7775-c12b-4067-98fc-6e7e6fc342f7 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:03:39.530094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:39.598660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da19b7-d31d-4490-89e1-b43ecf6a670f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:03:39.599447 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da19b7-d31d-4490-89e1-b43ecf6a670f service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:39.599834 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da19b7-d31d-4490-89e1-b43ecf6a670f service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:39.600304 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da19b7-d31d-4490-89e1-b43ecf6a670f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:39.602484 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-42da19b7-d31d-4490-89e1-b43ecf6a670f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:39.712107 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da19b7-d31d-4490-89e1-b43ecf6a670f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:03:39.713848 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da19b7-d31d-4490-89e1-b43ecf6a670f service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:03:39.735069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 689/1385] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:03:39 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:03:41.808233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2da2b5-057b-42b2-8b8f-6917b9fd235d None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:41.808922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2da2b5-057b-42b2-8b8f-6917b9fd235d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:41.808922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2da2b5-057b-42b2-8b8f-6917b9fd235d None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:41.810170 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 697/1386] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:03:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 09:03:41.817421 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610dd42-a93c-431e-847e-df289b2252ca None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:41.817879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610dd42-a93c-431e-847e-df289b2252ca None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:41.818003 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610dd42-a93c-431e-847e-df289b2252ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:41.889792 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-4610dd42-a93c-431e-847e-df289b2252ca None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:41.891533 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-4610dd42-a93c-431e-847e-df289b2252ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:41.925312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:42.012388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 690/1387] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:03:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:03:42.059122 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2da2b5-057b-42b2-8b8f-6917b9fd235d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:03:42.060657 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9d2da2b5-057b-42b2-8b8f-6917b9fd235d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:42.224342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2da2b5-057b-42b2-8b8f-6917b9fd235d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:03:42.313539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:42.372345 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96fff1-bf4f-48d3-bb65-35e4553f081b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:03:42.373206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96fff1-bf4f-48d3-bb65-35e4553f081b service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:42.373566 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96fff1-bf4f-48d3-bb65-35e4553f081b service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:42.373924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96fff1-bf4f-48d3-bb65-35e4553f081b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:42.376066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7d96fff1-bf4f-48d3-bb65-35e4553f081b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:42.495026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96fff1-bf4f-48d3-bb65-35e4553f081b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:03:42.497069 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96fff1-bf4f-48d3-bb65-35e4553f081b service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:03:42.518883 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 698/1388] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:03:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:03:44.680964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a8ee2-035e-4bd2-8bfb-700ec17e4b87 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:44.681557 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a8ee2-035e-4bd2-8bfb-700ec17e4b87 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:44.681557 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a8ee2-035e-4bd2-8bfb-700ec17e4b87 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:44.682922 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 691/1389] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:03:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:44.688253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa815cb-ccf2-4ee9-a8c6-56b1e6028464 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:44.688713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa815cb-ccf2-4ee9-a8c6-56b1e6028464 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:44.688823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa815cb-ccf2-4ee9-a8c6-56b1e6028464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:44.760828 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ffa815cb-ccf2-4ee9-a8c6-56b1e6028464 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:44.762083 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ffa815cb-ccf2-4ee9-a8c6-56b1e6028464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:44.798770 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:44.880369 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 699/1390] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:03:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:03:44.926568 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a8ee2-035e-4bd2-8bfb-700ec17e4b87 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:03:44.928083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-5b1a8ee2-035e-4bd2-8bfb-700ec17e4b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:45.087070 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a8ee2-035e-4bd2-8bfb-700ec17e4b87 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:03:45.096292 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:45.206894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497581a7-1801-4ca3-80f2-79866abf8db9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:03:45.207508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497581a7-1801-4ca3-80f2-79866abf8db9 service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:45.207789 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497581a7-1801-4ca3-80f2-79866abf8db9 service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:45.208056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497581a7-1801-4ca3-80f2-79866abf8db9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:45.209687 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-497581a7-1801-4ca3-80f2-79866abf8db9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:45.306861 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497581a7-1801-4ca3-80f2-79866abf8db9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:03:45.308672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497581a7-1801-4ca3-80f2-79866abf8db9 service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:03:45.327420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 692/1391] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:03:44 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:03:46.347602 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7cc5c-5cc2-4074-86c1-e85ece1706a8 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:46.348221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7cc5c-5cc2-4074-86c1-e85ece1706a8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:46.348221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7cc5c-5cc2-4074-86c1-e85ece1706a8 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:46.349577 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 700/1392] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:03:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:46.354845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e209b6-ee11-43b8-a148-f506c96a900f None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:46.355315 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e209b6-ee11-43b8-a148-f506c96a900f None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:46.355421 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e209b6-ee11-43b8-a148-f506c96a900f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:46.371681 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:46.428639 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-84e209b6-ee11-43b8-a148-f506c96a900f None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:46.430014 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-84e209b6-ee11-43b8-a148-f506c96a900f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:46.535118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 693/1393] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:03:46 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:03:46.591800 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e71fe-ec65-4357-9f7b-943c122403c1 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:46.592095 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e71fe-ec65-4357-9f7b-943c122403c1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:46.592370 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e71fe-ec65-4357-9f7b-943c122403c1 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:46.593770 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 701/1394] 10.222.0.100 () {54 vars in 971 bytes} [Tue Aug 2 09:03:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:46.599033 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfe4ec-6082-4124-9925-3644f2dadb45 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:46.599243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfe4ec-6082-4124-9925-3644f2dadb45 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:46.599522 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfe4ec-6082-4124-9925-3644f2dadb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:46.677786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-accfe4ec-6082-4124-9925-3644f2dadb45 None None] MFA Rules not processed for user `d959ddd087724547a11c3d4c796d8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:46.678996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-accfe4ec-6082-4124-9925-3644f2dadb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:46.713839 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:46.760167 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 694/1395] 10.222.0.100 () {58 vars in 1068 bytes} [Tue Aug 2 09:03:46 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:03:46.773673 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e71fe-ec65-4357-9f7b-943c122403c1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:03:46.775273 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fe9e71fe-ec65-4357-9f7b-943c122403c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:46.888564 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e71fe-ec65-4357-9f7b-943c122403c1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:03:46.967418 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:46.983364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aefa4a-2e5b-4b7c-ae41-112166fdeb39 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd959ddd087724547a11c3d4c796d8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:03:46.983986 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aefa4a-2e5b-4b7c-ae41-112166fdeb39 service glance] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:46.984264 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aefa4a-2e5b-4b7c-ae41-112166fdeb39 service glance] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:46.984537 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aefa4a-2e5b-4b7c-ae41-112166fdeb39 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:46.986184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d5aefa4a-2e5b-4b7c-ae41-112166fdeb39 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:47.101672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aefa4a-2e5b-4b7c-ae41-112166fdeb39 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:03:47.103106 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aefa4a-2e5b-4b7c-ae41-112166fdeb39 service glance] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:03:47.123321 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 702/1396] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:03:46 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:03:47.419183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca3a7c-4024-45ca-822e-7475362ac64c None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:47.419839 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca3a7c-4024-45ca-822e-7475362ac64c None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:47.419839 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca3a7c-4024-45ca-822e-7475362ac64c None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:47.421244 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 695/1397] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:03:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:47.427232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f6c42-5a39-4644-8537-33c882cbde6f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:47.427817 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f6c42-5a39-4644-8537-33c882cbde6f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:47.427817 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f6c42-5a39-4644-8537-33c882cbde6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:47.507861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9d5f6c42-5a39-4644-8537-33c882cbde6f None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:47.509749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9d5f6c42-5a39-4644-8537-33c882cbde6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:47.561541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:47.623206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 703/1398] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:03:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:03:47.671103 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca3a7c-4024-45ca-822e-7475362ac64c None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:03:47.672680 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-89ca3a7c-4024-45ca-822e-7475362ac64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:47.851028 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca3a7c-4024-45ca-822e-7475362ac64c None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:03:47.900056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:47.922787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942bb25-1fe6-409f-8b04-094304fd3b17 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:47.922906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942bb25-1fe6-409f-8b04-094304fd3b17 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:47.923153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942bb25-1fe6-409f-8b04-094304fd3b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:47.988485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af908ec-df74-4373-af37-df9375aa86ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:03:47.989179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af908ec-df74-4373-af37-df9375aa86ee service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:47.989432 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af908ec-df74-4373-af37-df9375aa86ee service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:47.989752 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af908ec-df74-4373-af37-df9375aa86ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:47.991776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-2af908ec-df74-4373-af37-df9375aa86ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:47.995796 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c942bb25-1fe6-409f-8b04-094304fd3b17 None None] MFA Rules not processed for user `1cd36b543bd9487ebd8ce578e9a76c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:47.996991 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c942bb25-1fe6-409f-8b04-094304fd3b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:48.098153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 704/1399] 10.222.0.100 () {58 vars in 1014 bytes} [Tue Aug 2 09:03:47 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:03:48.098621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af908ec-df74-4373-af37-df9375aa86ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:03:48.100242 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af908ec-df74-4373-af37-df9375aa86ee service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:03:48.121739 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 696/1400] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:03:47 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:03:48.155215 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ffc15-6f5e-406e-a2bf-4ba61c8bfdb6 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:48.155641 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ffc15-6f5e-406e-a2bf-4ba61c8bfdb6 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:48.156029 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ffc15-6f5e-406e-a2bf-4ba61c8bfdb6 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:48.157554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 705/1401] 10.222.0.100 () {54 vars in 948 bytes} [Tue Aug 2 09:03:48 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 09:03:48.164724 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44086b-7271-4747-aab9-eb0602726e7b None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:48.165014 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44086b-7271-4747-aab9-eb0602726e7b None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:48.165350 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44086b-7271-4747-aab9-eb0602726e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:48.229242 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:48.263901 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-6b44086b-7271-4747-aab9-eb0602726e7b None None] MFA Rules not processed for user `50cb532b58ed4951ba960a0a7d579b49`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:48.265533 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-6b44086b-7271-4747-aab9-eb0602726e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:48.398579 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 697/1402] 10.222.0.100 () {58 vars in 1045 bytes} [Tue Aug 2 09:03:48 2022] POST /identity/v3/auth/tokens => generated 3314 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:03:48.408070 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ffc15-6f5e-406e-a2bf-4ba61c8bfdb6 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:03:48.409649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-df7ffc15-6f5e-406e-a2bf-4ba61c8bfdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:48.452555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:48.556846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ffc15-6f5e-406e-a2bf-4ba61c8bfdb6 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:03:48.666444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc85bfa-e7f1-4850-b65e-badb84b461f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50cb532b58ed4951ba960a0a7d579b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:03:48.666987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc85bfa-e7f1-4850-b65e-badb84b461f5 service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:48.667270 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc85bfa-e7f1-4850-b65e-badb84b461f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:48.667545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc85bfa-e7f1-4850-b65e-badb84b461f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:48.669281 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-dcc85bfa-e7f1-4850-b65e-badb84b461f5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:48.694562 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:48.765137 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc85bfa-e7f1-4850-b65e-badb84b461f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:03:48.766747 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc85bfa-e7f1-4850-b65e-badb84b461f5 service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:03:48.778789 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 706/1403] 10.222.0.100 () {60 vars in 1389 bytes} [Tue Aug 2 09:03:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 02 09:03:50.271798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b09f8-bc72-48c5-80a9-d140e15010f4 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:50.272380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b09f8-bc72-48c5-80a9-d140e15010f4 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:50.272380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b09f8-bc72-48c5-80a9-d140e15010f4 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:50.280202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635a8d0-c2b8-47d3-b2c0-95a0ca715f5a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:50.280202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635a8d0-c2b8-47d3-b2c0-95a0ca715f5a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:50.280202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635a8d0-c2b8-47d3-b2c0-95a0ca715f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:50.281342 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 698/1404] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:03:50 2022] GET /identity => generated 271 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:50.360413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-2635a8d0-c2b8-47d3-b2c0-95a0ca715f5a None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:50.362364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2635a8d0-c2b8-47d3-b2c0-95a0ca715f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:50.478739 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 707/1405] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:03:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:03:50.525344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b09f8-bc72-48c5-80a9-d140e15010f4 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:03:50.526895 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e23b09f8-bc72-48c5-80a9-d140e15010f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:50.662088 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:50.714925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b09f8-bc72-48c5-80a9-d140e15010f4 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:03:50.887668 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e648b-9f7a-4a71-89dc-05d6bd43cece service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:03:50.888325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e648b-9f7a-4a71-89dc-05d6bd43cece service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:50.888612 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e648b-9f7a-4a71-89dc-05d6bd43cece service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:50.888900 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e648b-9f7a-4a71-89dc-05d6bd43cece service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:50.890619 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-349e648b-9f7a-4a71-89dc-05d6bd43cece service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:50.983702 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:51.009885 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e648b-9f7a-4a71-89dc-05d6bd43cece service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:03:51.011921 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e648b-9f7a-4a71-89dc-05d6bd43cece service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:03:51.032457 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 699/1406] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:03:50 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:03:53.113969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998df944-8abe-4ae3-bd9e-d5a85f6fa280 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:53.114525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998df944-8abe-4ae3-bd9e-d5a85f6fa280 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:53.114525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998df944-8abe-4ae3-bd9e-d5a85f6fa280 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:53.116323 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 708/1407] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:03:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:03:53.121925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbaacf-f762-4d2e-8aca-c3a3a099d9b9 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:53.122396 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbaacf-f762-4d2e-8aca-c3a3a099d9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:53.122496 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbaacf-f762-4d2e-8aca-c3a3a099d9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:53.200493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-60cbaacf-f762-4d2e-8aca-c3a3a099d9b9 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:03:53.201685 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-60cbaacf-f762-4d2e-8aca-c3a3a099d9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:53.322434 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 700/1408] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:03:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:03:53.369528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998df944-8abe-4ae3-bd9e-d5a85f6fa280 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:03:53.371193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-998df944-8abe-4ae3-bd9e-d5a85f6fa280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:53.387271 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:53.565172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998df944-8abe-4ae3-bd9e-d5a85f6fa280 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:03:53.670188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:03:53.699413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a1e68-1b9f-4831-94fe-87954f0a72d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:03:53.700037 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a1e68-1b9f-4831-94fe-87954f0a72d5 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:03:53.700479 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a1e68-1b9f-4831-94fe-87954f0a72d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:03:53.700749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a1e68-1b9f-4831-94fe-87954f0a72d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:03:53.702441 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-495a1e68-1b9f-4831-94fe-87954f0a72d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:03:53.808058 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a1e68-1b9f-4831-94fe-87954f0a72d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:03:53.809760 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a1e68-1b9f-4831-94fe-87954f0a72d5 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:03:53.832925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 709/1409] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:03:53 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:03.320303 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=81 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:03.829499 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=56 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:12.160632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12dc5b-7980-4089-a60d-8576fece47d4 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:12.161251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12dc5b-7980-4089-a60d-8576fece47d4 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:12.161251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12dc5b-7980-4089-a60d-8576fece47d4 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:12.162565 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 701/1410] 10.222.0.100 () {54 vars in 923 bytes} [Tue Aug 2 09:04:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:04:12.167919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba5cb1-ea73-4b98-a3af-1a46e6ee55dd None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:12.168414 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba5cb1-ea73-4b98-a3af-1a46e6ee55dd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:12.168555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba5cb1-ea73-4b98-a3af-1a46e6ee55dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:12.250975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-efba5cb1-ea73-4b98-a3af-1a46e6ee55dd None None] MFA Rules not processed for user `b40981fe179a43dc81553d3594dad649`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:12.252988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-efba5cb1-ea73-4b98-a3af-1a46e6ee55dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:12.363498 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 710/1411] 10.222.0.100 () {58 vars in 1020 bytes} [Tue Aug 2 09:04:12 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:04:12.383270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b43c6-0843-489d-9889-6c947552d6ff None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:12.383559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b43c6-0843-489d-9889-6c947552d6ff None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:12.383867 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b43c6-0843-489d-9889-6c947552d6ff None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:12.385255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 702/1412] 10.222.0.100 () {54 vars in 923 bytes} [Tue Aug 2 09:04:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:04:12.390220 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a0c19-65e5-4be2-a596-a06420917c4e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:12.390669 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a0c19-65e5-4be2-a596-a06420917c4e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:12.391342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a0c19-65e5-4be2-a596-a06420917c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:12.469435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:12.473151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c72a0c19-65e5-4be2-a596-a06420917c4e None None] MFA Rules not processed for user `49a0c1a4a6cf43e28ec1b8488c681848`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:12.474537 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c72a0c19-65e5-4be2-a596-a06420917c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:12.604712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 711/1413] 10.222.0.100 () {58 vars in 1020 bytes} [Tue Aug 2 09:04:12 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:04:15.151620 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc1dcd-3e9d-4770-b5d4-0a931a252882 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:15.152185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc1dcd-3e9d-4770-b5d4-0a931a252882 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:15.152372 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc1dcd-3e9d-4770-b5d4-0a931a252882 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:15.153647 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:04:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:04:15.159060 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb630346-83dd-4961-bafa-da1e50e4faf2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:15.159497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb630346-83dd-4961-bafa-da1e50e4faf2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:15.159606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb630346-83dd-4961-bafa-da1e50e4faf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:15.228176 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-bb630346-83dd-4961-bafa-da1e50e4faf2 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:15.229333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bb630346-83dd-4961-bafa-da1e50e4faf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:15.292445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:15.336315 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:04:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:15.380270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc1dcd-3e9d-4770-b5d4-0a931a252882 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:15.381824 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f0cc1dcd-3e9d-4770-b5d4-0a931a252882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:15.682505 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc1dcd-3e9d-4770-b5d4-0a931a252882 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:15.770463 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:15.820023 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b64fa0-d014-4bb6-8886-2f52a67856ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:15.820759 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b64fa0-d014-4bb6-8886-2f52a67856ef service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:15.821066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b64fa0-d014-4bb6-8886-2f52a67856ef service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:15.821411 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b64fa0-d014-4bb6-8886-2f52a67856ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:15.823399 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e8b64fa0-d014-4bb6-8886-2f52a67856ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:15.928410 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b64fa0-d014-4bb6-8886-2f52a67856ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:15.930121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b64fa0-d014-4bb6-8886-2f52a67856ef service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:15.950018 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 704/1416] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:04:15 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:17.005257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf082f0-5bad-4d93-b373-67ce14f0c3eb None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:17.005887 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf082f0-5bad-4d93-b373-67ce14f0c3eb None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:17.006000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf082f0-5bad-4d93-b373-67ce14f0c3eb None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:17.007751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:04:17 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:04:17.013681 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d71e80-a310-4021-bbc0-eca3d2de337b None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:17.014251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d71e80-a310-4021-bbc0-eca3d2de337b None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:17.014251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d71e80-a310-4021-bbc0-eca3d2de337b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:17.095215 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-a2d71e80-a310-4021-bbc0-eca3d2de337b None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:17.096493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-a2d71e80-a310-4021-bbc0-eca3d2de337b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:17.126124 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:17.200740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:04:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:04:17.354441 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf082f0-5bad-4d93-b373-67ce14f0c3eb None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:17.356051 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-caf082f0-5bad-4d93-b373-67ce14f0c3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:17.520896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf082f0-5bad-4d93-b373-67ce14f0c3eb None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:17.563536 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:17.645753 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6b576-d009-4f46-b952-4af0ca9de6af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:17.646379 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6b576-d009-4f46-b952-4af0ca9de6af service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:17.646660 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6b576-d009-4f46-b952-4af0ca9de6af service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:17.646932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6b576-d009-4f46-b952-4af0ca9de6af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:17.648563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e3d6b576-d009-4f46-b952-4af0ca9de6af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:17.742147 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6b576-d009-4f46-b952-4af0ca9de6af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:17.743772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6b576-d009-4f46-b952-4af0ca9de6af service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:17.762067 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 714/1419] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:04:17 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:18.800381 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51056a1-4bce-45ce-aff8-b11675a1e79e None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:18.801029 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51056a1-4bce-45ce-aff8-b11675a1e79e None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:18.801029 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51056a1-4bce-45ce-aff8-b11675a1e79e None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:18.802357 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:04:18 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 09:04:18.807782 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a491420-e4d7-4db7-8c4a-fb9e26971eb4 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:18.808268 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a491420-e4d7-4db7-8c4a-fb9e26971eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:18.808448 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a491420-e4d7-4db7-8c4a-fb9e26971eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:18.872535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:18.895039 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-0a491420-e4d7-4db7-8c4a-fb9e26971eb4 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:18.896252 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0a491420-e4d7-4db7-8c4a-fb9e26971eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:19.010507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:04:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:04:19.172433 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51056a1-4bce-45ce-aff8-b11675a1e79e None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:19.174933 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-a51056a1-4bce-45ce-aff8-b11675a1e79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:19.342428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51056a1-4bce-45ce-aff8-b11675a1e79e None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:19.355744 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:19.479405 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169c59a-25c7-4fb9-b204-cabfa34aa1ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:19.479933 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169c59a-25c7-4fb9-b204-cabfa34aa1ca service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:19.480211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169c59a-25c7-4fb9-b204-cabfa34aa1ca service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:19.480486 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169c59a-25c7-4fb9-b204-cabfa34aa1ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:19.482146 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-7169c59a-25c7-4fb9-b204-cabfa34aa1ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:19.588311 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169c59a-25c7-4fb9-b204-cabfa34aa1ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:19.589960 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169c59a-25c7-4fb9-b204-cabfa34aa1ca service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:19.608850 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 707/1422] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:04:19 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:20.512278 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef11d3e-1c7c-439f-b27b-43f96c60c45c None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:20.512828 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef11d3e-1c7c-439f-b27b-43f96c60c45c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:20.512828 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef11d3e-1c7c-439f-b27b-43f96c60c45c None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:20.514241 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:04:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:04:20.520072 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55ee08-1d98-4741-ba13-dfec3aa6ca60 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:20.520549 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55ee08-1d98-4741-ba13-dfec3aa6ca60 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:20.520727 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55ee08-1d98-4741-ba13-dfec3aa6ca60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:20.543054 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:20.606300 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-0e55ee08-1d98-4741-ba13-dfec3aa6ca60 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:20.607633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-0e55ee08-1d98-4741-ba13-dfec3aa6ca60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:20.719856 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:04:20 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:20.893877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef11d3e-1c7c-439f-b27b-43f96c60c45c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:20.895310 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-cef11d3e-1c7c-439f-b27b-43f96c60c45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:21.032013 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:21.077785 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef11d3e-1c7c-439f-b27b-43f96c60c45c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:21.208423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feec62b-e90f-4d5d-997e-656e6c1101e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:21.208968 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feec62b-e90f-4d5d-997e-656e6c1101e2 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:21.209294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feec62b-e90f-4d5d-997e-656e6c1101e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:21.209575 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feec62b-e90f-4d5d-997e-656e6c1101e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:21.211262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8feec62b-e90f-4d5d-997e-656e6c1101e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:21.295726 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:21.318090 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feec62b-e90f-4d5d-997e-656e6c1101e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:21.319730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feec62b-e90f-4d5d-997e-656e6c1101e2 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:21.339144 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 717/1425] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:04:20 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:22.238701 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aec630-3341-4428-b5aa-54c85e80f3e1 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:22.239233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aec630-3341-4428-b5aa-54c85e80f3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:22.239233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aec630-3341-4428-b5aa-54c85e80f3e1 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:22.240560 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:04:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:04:22.245796 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418b22e-5385-4aa9-a8d6-b72c48e31590 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:22.246221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418b22e-5385-4aa9-a8d6-b72c48e31590 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:22.246363 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418b22e-5385-4aa9-a8d6-b72c48e31590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:22.325714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f418b22e-5385-4aa9-a8d6-b72c48e31590 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:22.327521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f418b22e-5385-4aa9-a8d6-b72c48e31590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:22.432616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:04:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:22.607195 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aec630-3341-4428-b5aa-54c85e80f3e1 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:22.608712 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b7aec630-3341-4428-b5aa-54c85e80f3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:22.703146 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:22.785884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aec630-3341-4428-b5aa-54c85e80f3e1 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:22.917210 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769b4aa-236b-4cb3-8645-a11bc4dcb91c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:22.917731 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769b4aa-236b-4cb3-8645-a11bc4dcb91c service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:22.918009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769b4aa-236b-4cb3-8645-a11bc4dcb91c service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:22.918278 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769b4aa-236b-4cb3-8645-a11bc4dcb91c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:22.919957 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b769b4aa-236b-4cb3-8645-a11bc4dcb91c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:22.977125 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:23.027556 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769b4aa-236b-4cb3-8645-a11bc4dcb91c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:23.029160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769b4aa-236b-4cb3-8645-a11bc4dcb91c service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:23.051329 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 710/1428] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:04:22 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:23.891128 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d71b2-1ad9-4d78-9569-26a1a712838c None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:23.891687 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d71b2-1ad9-4d78-9569-26a1a712838c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:23.891687 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d71b2-1ad9-4d78-9569-26a1a712838c None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:23.893616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:04:23 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:04:23.899401 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0f24f-9af0-4edd-9970-d3d42709c44a None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:23.899928 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0f24f-9af0-4edd-9970-d3d42709c44a None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:23.899928 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0f24f-9af0-4edd-9970-d3d42709c44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:23.982279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-e6f0f24f-9af0-4edd-9970-d3d42709c44a None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:23.983502 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e6f0f24f-9af0-4edd-9970-d3d42709c44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:24.095578 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:04:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:24.249444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d71b2-1ad9-4d78-9569-26a1a712838c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:24.250922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-811d71b2-1ad9-4d78-9569-26a1a712838c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:24.303863 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:24.413902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d71b2-1ad9-4d78-9569-26a1a712838c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:24.538018 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca145f96-8020-4bfe-9b26-6f05cc2ad3a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:24.538657 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca145f96-8020-4bfe-9b26-6f05cc2ad3a6 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:24.538942 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca145f96-8020-4bfe-9b26-6f05cc2ad3a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:24.539386 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca145f96-8020-4bfe-9b26-6f05cc2ad3a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:24.541056 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ca145f96-8020-4bfe-9b26-6f05cc2ad3a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:24.548379 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:24.763486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca145f96-8020-4bfe-9b26-6f05cc2ad3a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:24.765080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca145f96-8020-4bfe-9b26-6f05cc2ad3a6 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:24.784953 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 720/1431] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:04:24 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:25.693356 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf50a2-83cd-4b5c-b8d9-9eb33a64d486 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:25.693937 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf50a2-83cd-4b5c-b8d9-9eb33a64d486 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:25.693937 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf50a2-83cd-4b5c-b8d9-9eb33a64d486 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:25.695411 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:04:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:04:25.700768 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc3a9a-f59e-4171-96b9-af92f1daf3db None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:25.701329 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc3a9a-f59e-4171-96b9-af92f1daf3db None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:25.701329 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc3a9a-f59e-4171-96b9-af92f1daf3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:25.792993 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-72dc3a9a-f59e-4171-96b9-af92f1daf3db None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:25.794199 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-72dc3a9a-f59e-4171-96b9-af92f1daf3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:25.875351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:25.907019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:04:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:26.077693 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf50a2-83cd-4b5c-b8d9-9eb33a64d486 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:26.079120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-4bdf50a2-83cd-4b5c-b8d9-9eb33a64d486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:26.099751 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:26.263050 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf50a2-83cd-4b5c-b8d9-9eb33a64d486 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:26.356774 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:26.385285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f93e6-5755-4bbb-aefb-7b1ee85e986b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:26.385904 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f93e6-5755-4bbb-aefb-7b1ee85e986b service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:26.386183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f93e6-5755-4bbb-aefb-7b1ee85e986b service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:26.386452 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f93e6-5755-4bbb-aefb-7b1ee85e986b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:26.388066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1e0f93e6-5755-4bbb-aefb-7b1ee85e986b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:26.482426 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f93e6-5755-4bbb-aefb-7b1ee85e986b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:26.484088 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f93e6-5755-4bbb-aefb-7b1ee85e986b service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:26.503418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 713/1434] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:04:26 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:27.357497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451f694-c09d-435d-9427-404835f15788 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:27.358110 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451f694-c09d-435d-9427-404835f15788 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:27.358110 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451f694-c09d-435d-9427-404835f15788 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:27.359433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:04:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:04:27.364555 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028d321-8cc1-49d2-831b-0e0723fa3584 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:27.365027 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028d321-8cc1-49d2-831b-0e0723fa3584 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:27.365177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028d321-8cc1-49d2-831b-0e0723fa3584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:27.453454 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-5028d321-8cc1-49d2-831b-0e0723fa3584 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:27.454667 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-5028d321-8cc1-49d2-831b-0e0723fa3584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:27.504082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:27.562340 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:04:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:27.722503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451f694-c09d-435d-9427-404835f15788 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:27.724072 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8451f694-c09d-435d-9427-404835f15788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:27.893494 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451f694-c09d-435d-9427-404835f15788 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:27.962251 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:28.027007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760f139-c80c-4cb3-964f-092f9b1e9649 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:28.027636 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760f139-c80c-4cb3-964f-092f9b1e9649 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:28.027905 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760f139-c80c-4cb3-964f-092f9b1e9649 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:28.028195 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760f139-c80c-4cb3-964f-092f9b1e9649 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:28.029805 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f760f139-c80c-4cb3-964f-092f9b1e9649 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:28.125949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760f139-c80c-4cb3-964f-092f9b1e9649 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:28.127738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760f139-c80c-4cb3-964f-092f9b1e9649 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:28.146909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 723/1437] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:04:27 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:28.993175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eb970-2717-4f3d-9ff0-743f67fd6e87 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:28.993711 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eb970-2717-4f3d-9ff0-743f67fd6e87 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:28.993835 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eb970-2717-4f3d-9ff0-743f67fd6e87 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:28.995183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:04:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:04:29.000496 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85bd66-5426-4f70-b84a-ecd9614af2c3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:29.000918 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85bd66-5426-4f70-b84a-ecd9614af2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:29.001043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85bd66-5426-4f70-b84a-ecd9614af2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:29.094707 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9b85bd66-5426-4f70-b84a-ecd9614af2c3 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:29.096743 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9b85bd66-5426-4f70-b84a-ecd9614af2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:29.098955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:29.202938 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:04:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:29.362196 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58eb970-2717-4f3d-9ff0-743f67fd6e87 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:29.364142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e58eb970-2717-4f3d-9ff0-743f67fd6e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:29.526996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58eb970-2717-4f3d-9ff0-743f67fd6e87 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:29.557217 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:29.651285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da791c6-9d1b-4f41-b2a1-44fb6b919cbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:29.651912 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da791c6-9d1b-4f41-b2a1-44fb6b919cbe service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:29.652194 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da791c6-9d1b-4f41-b2a1-44fb6b919cbe service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:29.652469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da791c6-9d1b-4f41-b2a1-44fb6b919cbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:29.654158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3da791c6-9d1b-4f41-b2a1-44fb6b919cbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:29.753102 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da791c6-9d1b-4f41-b2a1-44fb6b919cbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:29.754752 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da791c6-9d1b-4f41-b2a1-44fb6b919cbe service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:29.773630 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 716/1440] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:04:29 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:31.752000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ca1cd-2264-487e-8932-7707a5e8ef5b None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:31.753165 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ca1cd-2264-487e-8932-7707a5e8ef5b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:31.753165 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ca1cd-2264-487e-8932-7707a5e8ef5b None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:31.754624 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:04:31 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:04:31.760409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42f9c7-07f7-442c-aa31-43d4739fb6cf None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:31.760910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42f9c7-07f7-442c-aa31-43d4739fb6cf None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:31.760910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42f9c7-07f7-442c-aa31-43d4739fb6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:31.806735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:31.835168 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-2a42f9c7-07f7-442c-aa31-43d4739fb6cf None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:31.836390 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-2a42f9c7-07f7-442c-aa31-43d4739fb6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:31.944293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:04:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:31.984512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ca1cd-2264-487e-8932-7707a5e8ef5b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:31.986118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5f9ca1cd-2264-487e-8932-7707a5e8ef5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:32.101865 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ca1cd-2264-487e-8932-7707a5e8ef5b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:32.135974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:32.181581 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe83a7-a2d1-4e52-8a01-ab6f7f1e55fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:32.182220 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe83a7-a2d1-4e52-8a01-ab6f7f1e55fb service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:32.182514 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe83a7-a2d1-4e52-8a01-ab6f7f1e55fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:32.182790 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe83a7-a2d1-4e52-8a01-ab6f7f1e55fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:32.184447 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2dfe83a7-a2d1-4e52-8a01-ab6f7f1e55fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:32.304576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe83a7-a2d1-4e52-8a01-ab6f7f1e55fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:32.306049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe83a7-a2d1-4e52-8a01-ab6f7f1e55fb service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:32.325132 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 726/1443] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:04:31 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:04:34.646002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e2102-be10-4f8a-bf4b-4f0b0dd5af9a None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:34.646595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e2102-be10-4f8a-bf4b-4f0b0dd5af9a None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:34.646595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e2102-be10-4f8a-bf4b-4f0b0dd5af9a None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:34.647933 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:04:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:04:34.653851 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9277abc-0e8f-4403-ab91-67db7ed74b02 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:34.654358 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9277abc-0e8f-4403-ab91-67db7ed74b02 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:34.654469 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9277abc-0e8f-4403-ab91-67db7ed74b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:34.727544 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a9277abc-0e8f-4403-ab91-67db7ed74b02 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:34.728808 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a9277abc-0e8f-4403-ab91-67db7ed74b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:34.743510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:34.836608 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:04:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:34.876551 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e2102-be10-4f8a-bf4b-4f0b0dd5af9a None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:34.880598 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-7d0e2102-be10-4f8a-bf4b-4f0b0dd5af9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:34.995445 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e2102-be10-4f8a-bf4b-4f0b0dd5af9a None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:35.009404 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:35.079107 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fee1de2-445e-40cd-822e-c2b6e24e2c5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:35.079734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee1de2-445e-40cd-822e-c2b6e24e2c5a service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:35.080016 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee1de2-445e-40cd-822e-c2b6e24e2c5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:35.080288 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee1de2-445e-40cd-822e-c2b6e24e2c5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:35.082344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3fee1de2-445e-40cd-822e-c2b6e24e2c5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:35.205879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fee1de2-445e-40cd-822e-c2b6e24e2c5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:35.207378 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fee1de2-445e-40cd-822e-c2b6e24e2c5a service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:35.227180 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 719/1446] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:04:34 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:37.789572 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf4c33-ff17-44a4-9b57-89e7abb14a54 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:37.790083 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf4c33-ff17-44a4-9b57-89e7abb14a54 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:37.790083 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf4c33-ff17-44a4-9b57-89e7abb14a54 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:37.791426 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:04:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:04:37.796779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8dc28-49a9-40df-a069-1968103cd95d None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:37.797206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8dc28-49a9-40df-a069-1968103cd95d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:37.797375 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8dc28-49a9-40df-a069-1968103cd95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:37.847761 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:37.866684 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-1cb8dc28-49a9-40df-a069-1968103cd95d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:37.868170 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1cb8dc28-49a9-40df-a069-1968103cd95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:37.978637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:04:37 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:38.018825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf4c33-ff17-44a4-9b57-89e7abb14a54 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:38.020349 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-95cf4c33-ff17-44a4-9b57-89e7abb14a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:38.125428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf4c33-ff17-44a4-9b57-89e7abb14a54 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:38.167088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:38.202355 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb4349-7e2e-414c-a156-1881453593b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:38.202982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb4349-7e2e-414c-a156-1881453593b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:38.203267 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb4349-7e2e-414c-a156-1881453593b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:38.203540 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb4349-7e2e-414c-a156-1881453593b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:38.205192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b1cb4349-7e2e-414c-a156-1881453593b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:38.318112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb4349-7e2e-414c-a156-1881453593b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:38.319538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb4349-7e2e-414c-a156-1881453593b6 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:38.338410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 729/1449] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:04:38 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:40.515034 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09faa1-597d-4f34-bf20-6cc2be3af0ae None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:40.515577 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09faa1-597d-4f34-bf20-6cc2be3af0ae None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:40.515577 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09faa1-597d-4f34-bf20-6cc2be3af0ae None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:40.517010 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:04:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:04:40.522653 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695eef00-2fc2-4fa5-8205-a94a53f3cc70 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:40.523080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695eef00-2fc2-4fa5-8205-a94a53f3cc70 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:40.523186 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695eef00-2fc2-4fa5-8205-a94a53f3cc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:40.592367 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-695eef00-2fc2-4fa5-8205-a94a53f3cc70 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:40.593614 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-695eef00-2fc2-4fa5-8205-a94a53f3cc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:40.615452 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:40.696767 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:04:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:40.736434 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09faa1-597d-4f34-bf20-6cc2be3af0ae None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:40.738590 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-cf09faa1-597d-4f34-bf20-6cc2be3af0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:40.846576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09faa1-597d-4f34-bf20-6cc2be3af0ae None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:40.867032 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:40.925845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4cf95-8106-419c-b6d6-85bd2dd78cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:40.926473 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4cf95-8106-419c-b6d6-85bd2dd78cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:40.926759 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4cf95-8106-419c-b6d6-85bd2dd78cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:40.927032 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4cf95-8106-419c-b6d6-85bd2dd78cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:40.928675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-23e4cf95-8106-419c-b6d6-85bd2dd78cc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:41.050867 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4cf95-8106-419c-b6d6-85bd2dd78cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:41.052346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4cf95-8106-419c-b6d6-85bd2dd78cc1 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:41.071746 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 722/1452] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:04:40 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:43.207391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe931f83-0f99-4e8e-882f-4822f9ed6b98 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:43.207915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe931f83-0f99-4e8e-882f-4822f9ed6b98 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:43.207915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe931f83-0f99-4e8e-882f-4822f9ed6b98 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:43.209287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:04:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:04:43.215092 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d20031-2439-4959-840b-353ff205f7dc None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:43.215637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d20031-2439-4959-840b-353ff205f7dc None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:43.215637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d20031-2439-4959-840b-353ff205f7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:43.290932 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:43.293902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-67d20031-2439-4959-840b-353ff205f7dc None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:43.296786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-67d20031-2439-4959-840b-353ff205f7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:43.398056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:04:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:04:43.437538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe931f83-0f99-4e8e-882f-4822f9ed6b98 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:43.439121 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fe931f83-0f99-4e8e-882f-4822f9ed6b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:43.550816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe931f83-0f99-4e8e-882f-4822f9ed6b98 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:43.602453 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:43.629915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57323825-8474-4090-81a4-bab51c1b9d7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:43.630576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57323825-8474-4090-81a4-bab51c1b9d7e service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:43.630860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57323825-8474-4090-81a4-bab51c1b9d7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:43.631154 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57323825-8474-4090-81a4-bab51c1b9d7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:43.632804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-57323825-8474-4090-81a4-bab51c1b9d7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:43.744721 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57323825-8474-4090-81a4-bab51c1b9d7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:43.746194 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57323825-8474-4090-81a4-bab51c1b9d7e service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:43.765025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 732/1455] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:04:43 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:45.935962 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c5b7e-9ec3-442e-a975-0f5d9d80ca6b None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:45.936524 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c5b7e-9ec3-442e-a975-0f5d9d80ca6b None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:45.936524 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c5b7e-9ec3-442e-a975-0f5d9d80ca6b None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:45.937914 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:04:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:04:45.943452 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fca170-87c8-405f-bda3-f98757e7be94 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:45.943939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fca170-87c8-405f-bda3-f98757e7be94 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:45.944038 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fca170-87c8-405f-bda3-f98757e7be94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:46.013541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-87fca170-87c8-405f-bda3-f98757e7be94 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:46.014733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-87fca170-87c8-405f-bda3-f98757e7be94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:46.046465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:46.128887 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:04:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:46.168513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c5b7e-9ec3-442e-a975-0f5d9d80ca6b None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:46.170115 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3f6c5b7e-9ec3-442e-a975-0f5d9d80ca6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:46.283498 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c5b7e-9ec3-442e-a975-0f5d9d80ca6b None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:46.317189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:46.367745 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417f674-3358-4125-8799-e90f3d74fc09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:46.368374 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417f674-3358-4125-8799-e90f3d74fc09 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:46.368923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417f674-3358-4125-8799-e90f3d74fc09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:46.369226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417f674-3358-4125-8799-e90f3d74fc09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:46.370882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-8417f674-3358-4125-8799-e90f3d74fc09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:46.487803 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8417f674-3358-4125-8799-e90f3d74fc09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:46.489288 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8417f674-3358-4125-8799-e90f3d74fc09 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:46.512308 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 725/1458] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:04:46 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:48.655537 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b470fa-d8ba-44f9-b42b-9f9fb60fda42 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:48.656151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b470fa-d8ba-44f9-b42b-9f9fb60fda42 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:48.656151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b470fa-d8ba-44f9-b42b-9f9fb60fda42 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:48.657476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:04:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:04:48.663685 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f588d-fa64-44ce-b47a-95031eab6329 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:48.664230 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f588d-fa64-44ce-b47a-95031eab6329 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:48.664230 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f588d-fa64-44ce-b47a-95031eab6329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:48.757736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-1f2f588d-fa64-44ce-b47a-95031eab6329 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:48.758992 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1f2f588d-fa64-44ce-b47a-95031eab6329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:48.767925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:48.871886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:04:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:48.912176 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b470fa-d8ba-44f9-b42b-9f9fb60fda42 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:48.913703 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-07b470fa-d8ba-44f9-b42b-9f9fb60fda42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:49.033896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b470fa-d8ba-44f9-b42b-9f9fb60fda42 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:49.105477 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:49.124746 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31c9e1-c61f-436b-af0e-93a29d82ec96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:49.125404 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31c9e1-c61f-436b-af0e-93a29d82ec96 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:49.125719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31c9e1-c61f-436b-af0e-93a29d82ec96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:49.126044 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31c9e1-c61f-436b-af0e-93a29d82ec96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:49.127739 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-df31c9e1-c61f-436b-af0e-93a29d82ec96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:49.257077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31c9e1-c61f-436b-af0e-93a29d82ec96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:49.260979 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31c9e1-c61f-436b-af0e-93a29d82ec96 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:49.290503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 735/1461] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:04:48 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:04:51.491244 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f7c2b-45f9-4031-a545-66d32b8133e8 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:51.491791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f7c2b-45f9-4031-a545-66d32b8133e8 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:51.491915 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f7c2b-45f9-4031-a545-66d32b8133e8 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:51.493253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:04:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:04:51.499002 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b781e-4603-479d-825e-00d85f4b5609 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:51.501229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b781e-4603-479d-825e-00d85f4b5609 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:51.501229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b781e-4603-479d-825e-00d85f4b5609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:51.573472 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-248b781e-4603-479d-825e-00d85f4b5609 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:51.574906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-248b781e-4603-479d-825e-00d85f4b5609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:51.619587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:51.680039 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:04:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:51.717219 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f7c2b-45f9-4031-a545-66d32b8133e8 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:51.718768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-bc0f7c2b-45f9-4031-a545-66d32b8133e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:51.825501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f7c2b-45f9-4031-a545-66d32b8133e8 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:51.866847 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:51.904987 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37651309-ac1c-4ddf-983f-4e0461d312b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:51.905636 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37651309-ac1c-4ddf-983f-4e0461d312b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:51.905918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37651309-ac1c-4ddf-983f-4e0461d312b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:51.906185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37651309-ac1c-4ddf-983f-4e0461d312b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:51.907810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-37651309-ac1c-4ddf-983f-4e0461d312b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:52.033983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37651309-ac1c-4ddf-983f-4e0461d312b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:52.036846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37651309-ac1c-4ddf-983f-4e0461d312b7 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:52.061986 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 728/1464] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:04:51 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:54.176784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf388e-ee16-4568-b266-3fab1c99aed7 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:54.177371 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf388e-ee16-4568-b266-3fab1c99aed7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:54.177371 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf388e-ee16-4568-b266-3fab1c99aed7 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:54.178647 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:04:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:04:54.184735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb62b8d-4436-46e2-af08-800558d2a24a None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:54.185304 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb62b8d-4436-46e2-af08-800558d2a24a None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:54.185419 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb62b8d-4436-46e2-af08-800558d2a24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:54.260845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-dbb62b8d-4436-46e2-af08-800558d2a24a None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:54.262107 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-dbb62b8d-4436-46e2-af08-800558d2a24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:54.281668 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:54.366674 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:04:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:54.403668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf388e-ee16-4568-b266-3fab1c99aed7 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:54.405262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-acaf388e-ee16-4568-b266-3fab1c99aed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:54.519587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf388e-ee16-4568-b266-3fab1c99aed7 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:54.595627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:54.604416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ab141-2a27-47f2-a3d6-e6b5c49bd4f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:54.605044 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ab141-2a27-47f2-a3d6-e6b5c49bd4f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:54.605360 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ab141-2a27-47f2-a3d6-e6b5c49bd4f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:54.605662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ab141-2a27-47f2-a3d6-e6b5c49bd4f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:54.607296 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ef9ab141-2a27-47f2-a3d6-e6b5c49bd4f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:54.720819 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ab141-2a27-47f2-a3d6-e6b5c49bd4f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:54.723031 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ab141-2a27-47f2-a3d6-e6b5c49bd4f5 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:54.742265 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 738/1467] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:04:54 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:56.792491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe5668-8d83-4dc0-a921-005dffc2614b None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:56.793148 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe5668-8d83-4dc0-a921-005dffc2614b None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:56.793148 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe5668-8d83-4dc0-a921-005dffc2614b None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:56.794635 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:04:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:04:56.799996 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799af1a0-751d-44a2-9c80-a6cc9c8878c5 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:56.800603 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799af1a0-751d-44a2-9c80-a6cc9c8878c5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:56.800603 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799af1a0-751d-44a2-9c80-a6cc9c8878c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:56.870730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-799af1a0-751d-44a2-9c80-a6cc9c8878c5 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:56.871952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-799af1a0-751d-44a2-9c80-a6cc9c8878c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:56.922090 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:56.974373 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:04:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:57.011615 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe5668-8d83-4dc0-a921-005dffc2614b None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:57.013165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3abe5668-8d83-4dc0-a921-005dffc2614b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:57.116160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe5668-8d83-4dc0-a921-005dffc2614b None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:57.164259 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:57.190488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42542942-7078-4c87-9499-5dca0b0117b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:57.191699 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42542942-7078-4c87-9499-5dca0b0117b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:57.191997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42542942-7078-4c87-9499-5dca0b0117b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:57.192277 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42542942-7078-4c87-9499-5dca0b0117b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:57.193927 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-42542942-7078-4c87-9499-5dca0b0117b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:57.320487 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42542942-7078-4c87-9499-5dca0b0117b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:57.321972 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42542942-7078-4c87-9499-5dca0b0117b7 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:57.341089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 731/1470] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:04:57 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:05:06.971116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=22 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:05:07.338708 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=66 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:05:17.646100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc866bc-860e-4f87-805c-879f65e03e08 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:17.646667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc866bc-860e-4f87-805c-879f65e03e08 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:17.646667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc866bc-860e-4f87-805c-879f65e03e08 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:17.648051 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 740/1471] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:05:17 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:05:17.653362 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078b85e-002b-4503-9010-5ac333fa7071 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:17.653831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078b85e-002b-4503-9010-5ac333fa7071 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:17.653938 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078b85e-002b-4503-9010-5ac333fa7071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:17.724362 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-2078b85e-002b-4503-9010-5ac333fa7071 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:05:17.725682 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-2078b85e-002b-4503-9010-5ac333fa7071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:17.824590 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 732/1472] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:05:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:05:17.911878 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44492c8d-1f55-4fbc-94f3-7b2ae96759e9 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:17.912160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44492c8d-1f55-4fbc-94f3-7b2ae96759e9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:17.912437 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44492c8d-1f55-4fbc-94f3-7b2ae96759e9 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:17.913825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 741/1473] 10.222.0.100 () {54 vars in 971 bytes} [Tue Aug 2 09:05:17 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:05:17.919135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227cab8-ae0c-44f6-8f0a-a2cbe546d768 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:17.919429 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227cab8-ae0c-44f6-8f0a-a2cbe546d768 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:17.919712 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227cab8-ae0c-44f6-8f0a-a2cbe546d768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:17.996348 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-2227cab8-ae0c-44f6-8f0a-a2cbe546d768 None None] MFA Rules not processed for user `d959ddd087724547a11c3d4c796d8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:05:17.997601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-2227cab8-ae0c-44f6-8f0a-a2cbe546d768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:18.020298 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:05:18.076763 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 733/1474] 10.222.0.100 () {58 vars in 1068 bytes} [Tue Aug 2 09:05:17 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:05:18.084309 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44492c8d-1f55-4fbc-94f3-7b2ae96759e9 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:05:18.085860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-44492c8d-1f55-4fbc-94f3-7b2ae96759e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:18.192120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44492c8d-1f55-4fbc-94f3-7b2ae96759e9 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:05:18.267576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796d50b-aac5-4ef9-b621-bfef5325e1d4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd959ddd087724547a11c3d4c796d8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:05:18.268201 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796d50b-aac5-4ef9-b621-bfef5325e1d4 service glance] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:18.268485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796d50b-aac5-4ef9-b621-bfef5325e1d4 service glance] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:18.268762 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796d50b-aac5-4ef9-b621-bfef5325e1d4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:18.271438 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e796d50b-aac5-4ef9-b621-bfef5325e1d4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:18.327966 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:05:18.389980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796d50b-aac5-4ef9-b621-bfef5325e1d4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:05:18.391417 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796d50b-aac5-4ef9-b621-bfef5325e1d4 service glance] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:05:18.410918 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 742/1475] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:05:18 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:05:20.528293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bf76f-3e01-4073-9a2c-892ea0a1cfcc None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:20.528908 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bf76f-3e01-4073-9a2c-892ea0a1cfcc None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:20.528908 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bf76f-3e01-4073-9a2c-892ea0a1cfcc None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:20.531086 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 734/1476] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:05:20 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:05:20.536671 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b7512-4e9b-4099-992a-5802df679297 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:20.537188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b7512-4e9b-4099-992a-5802df679297 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:20.537288 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b7512-4e9b-4099-992a-5802df679297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:20.604658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d46b7512-4e9b-4099-992a-5802df679297 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:05:20.605913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d46b7512-4e9b-4099-992a-5802df679297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:20.704724 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 743/1477] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:05:20 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:05:20.727560 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bf76f-3e01-4073-9a2c-892ea0a1cfcc None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:05:20.729139 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-aa4bf76f-3e01-4073-9a2c-892ea0a1cfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:20.856817 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bf76f-3e01-4073-9a2c-892ea0a1cfcc None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:05:20.908037 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:05:20.957877 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cdb2b3-7104-472c-babd-ad17484e2f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:05:20.958504 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdb2b3-7104-472c-babd-ad17484e2f47 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:20.958786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdb2b3-7104-472c-babd-ad17484e2f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:20.959063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdb2b3-7104-472c-babd-ad17484e2f47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:20.959764 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cdb2b3-7104-472c-babd-ad17484e2f47 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:05:20.961564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: WARNING py.warnings [None req-89cdb2b3-7104-472c-babd-ad17484e2f47 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:05:20.961564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: warnings.warn(msg) Aug 02 09:05:20.961564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: Aug 02 09:05:20.962735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cdb2b3-7104-472c-babd-ad17484e2f47 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:05:20.983136 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 735/1478] 10.222.0.100 () {56 vars in 1113 bytes} [Tue Aug 2 09:05:20 2022] GET /identity/v3/projects => generated 2562 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:05:23.038638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74519507-de6e-495a-ac46-cffda97c6c46 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:23.039258 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74519507-de6e-495a-ac46-cffda97c6c46 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:23.039258 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74519507-de6e-495a-ac46-cffda97c6c46 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:23.040700 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 744/1479] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:05:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:05:23.047028 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1633824-a31c-47b8-b206-78c9d258e429 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:23.047839 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1633824-a31c-47b8-b206-78c9d258e429 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:23.047839 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1633824-a31c-47b8-b206-78c9d258e429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:23.147750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-d1633824-a31c-47b8-b206-78c9d258e429 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:05:23.148995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d1633824-a31c-47b8-b206-78c9d258e429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:23.270233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 736/1480] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:05:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:05:23.321641 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74519507-de6e-495a-ac46-cffda97c6c46 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:05:23.323471 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-74519507-de6e-495a-ac46-cffda97c6c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:23.336055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:05:23.526956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74519507-de6e-495a-ac46-cffda97c6c46 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:05:23.618835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:05:23.655209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a16091-c1b0-430b-bf3b-90f5dddefd6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:05:23.655915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a16091-c1b0-430b-bf3b-90f5dddefd6e service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:23.656271 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a16091-c1b0-430b-bf3b-90f5dddefd6e service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:23.656593 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a16091-c1b0-430b-bf3b-90f5dddefd6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:23.658482 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d4a16091-c1b0-430b-bf3b-90f5dddefd6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:23.757241 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a16091-c1b0-430b-bf3b-90f5dddefd6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:05:23.759038 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a16091-c1b0-430b-bf3b-90f5dddefd6e service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:05:23.778569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 745/1481] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:05:23 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:05:26.117779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938696b-9a55-4325-8dec-cf0c53800843 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:26.118449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938696b-9a55-4325-8dec-cf0c53800843 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:26.118449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938696b-9a55-4325-8dec-cf0c53800843 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:26.119742 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 737/1482] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:05:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:05:26.127228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7513a-7fee-4add-8d88-afa5d4e024e2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:26.127818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7513a-7fee-4add-8d88-afa5d4e024e2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:26.127974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7513a-7fee-4add-8d88-afa5d4e024e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:26.207014 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6ec7513a-7fee-4add-8d88-afa5d4e024e2 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:05:26.208253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6ec7513a-7fee-4add-8d88-afa5d4e024e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:26.271332 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:05:26.350314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 746/1483] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:05:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:05:26.391080 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938696b-9a55-4325-8dec-cf0c53800843 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:05:26.393480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-8938696b-9a55-4325-8dec-cf0c53800843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:26.427645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:05:26.538851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938696b-9a55-4325-8dec-cf0c53800843 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:05:26.640734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f1756-bc8d-44d7-ae8a-b39265677914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:05:26.641443 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f1756-bc8d-44d7-ae8a-b39265677914 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:26.641727 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f1756-bc8d-44d7-ae8a-b39265677914 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:26.642025 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f1756-bc8d-44d7-ae8a-b39265677914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:26.643687 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f89f1756-bc8d-44d7-ae8a-b39265677914 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:26.729130 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:05:26.769289 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f1756-bc8d-44d7-ae8a-b39265677914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:05:26.771609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f1756-bc8d-44d7-ae8a-b39265677914 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:05:26.792766 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 738/1484] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:05:26 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:05:29.013480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dae235-d866-4ae4-a300-7ecf559cf3dc None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:29.014053 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dae235-d866-4ae4-a300-7ecf559cf3dc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:29.014053 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dae235-d866-4ae4-a300-7ecf559cf3dc None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:29.015387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 747/1485] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:05:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 09:05:29.020955 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59bb46-d7b9-4570-97de-c7edc5c4c28d None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:29.021438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59bb46-d7b9-4570-97de-c7edc5c4c28d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:29.021541 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59bb46-d7b9-4570-97de-c7edc5c4c28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:29.097682 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-bc59bb46-d7b9-4570-97de-c7edc5c4c28d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:05:29.100104 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-bc59bb46-d7b9-4570-97de-c7edc5c4c28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:29.208632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 739/1486] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:05:29 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:05:29.252144 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dae235-d866-4ae4-a300-7ecf559cf3dc None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:05:29.253945 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f4dae235-d866-4ae4-a300-7ecf559cf3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:29.381952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dae235-d866-4ae4-a300-7ecf559cf3dc None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:05:29.500390 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:05:29.503455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8bb83-fd7b-47ac-8d33-32e564f4a76c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:05:29.505262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8bb83-fd7b-47ac-8d33-32e564f4a76c service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:29.505923 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8bb83-fd7b-47ac-8d33-32e564f4a76c service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:29.506537 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8bb83-fd7b-47ac-8d33-32e564f4a76c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:29.509188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c0f8bb83-fd7b-47ac-8d33-32e564f4a76c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:29.668555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8bb83-fd7b-47ac-8d33-32e564f4a76c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:05:29.670798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8bb83-fd7b-47ac-8d33-32e564f4a76c service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:05:29.689713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 748/1487] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:05:29 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:05:31.890373 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4cb4e-e2da-45cc-a2e3-450757de3a19 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:31.890913 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4cb4e-e2da-45cc-a2e3-450757de3a19 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:31.890913 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4cb4e-e2da-45cc-a2e3-450757de3a19 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:31.892389 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 740/1488] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:05:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:05:31.898890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b4916-a080-43b0-8d26-6aa2b5423812 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:31.899593 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b4916-a080-43b0-8d26-6aa2b5423812 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:31.899593 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b4916-a080-43b0-8d26-6aa2b5423812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:31.976276 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ce8b4916-a080-43b0-8d26-6aa2b5423812 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:05:31.977570 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ce8b4916-a080-43b0-8d26-6aa2b5423812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:32.050547 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:05:32.100207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 749/1489] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:05:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:05:32.143639 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4cb4e-e2da-45cc-a2e3-450757de3a19 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:05:32.146130 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-15a4cb4e-e2da-45cc-a2e3-450757de3a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:32.197296 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:05:32.268556 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4cb4e-e2da-45cc-a2e3-450757de3a19 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:05:32.346510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471dac7-4ac1-4c72-a5e9-cc799bf19ed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:05:32.347202 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471dac7-4ac1-4c72-a5e9-cc799bf19ed3 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:32.347527 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471dac7-4ac1-4c72-a5e9-cc799bf19ed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:32.347924 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471dac7-4ac1-4c72-a5e9-cc799bf19ed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:32.349749 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b471dac7-4ac1-4c72-a5e9-cc799bf19ed3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:32.440786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:05:32.474034 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471dac7-4ac1-4c72-a5e9-cc799bf19ed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:05:32.475491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471dac7-4ac1-4c72-a5e9-cc799bf19ed3 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:05:32.495239 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 741/1490] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:05:32 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:05:34.777265 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27775be4-a7e4-48f2-ac7e-efa39d00f5da None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:34.778381 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27775be4-a7e4-48f2-ac7e-efa39d00f5da None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:34.778381 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27775be4-a7e4-48f2-ac7e-efa39d00f5da None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:34.779208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 750/1491] 10.222.0.100 () {54 vars in 931 bytes} [Tue Aug 2 09:05:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:05:34.784933 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe00310-6991-4fb4-bfe4-c64c0e980951 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:34.785496 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe00310-6991-4fb4-bfe4-c64c0e980951 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:34.785642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe00310-6991-4fb4-bfe4-c64c0e980951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:34.861228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-bfe00310-6991-4fb4-bfe4-c64c0e980951 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:05:34.862464 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-bfe00310-6991-4fb4-bfe4-c64c0e980951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:34.974094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 742/1492] 10.222.0.100 () {58 vars in 1028 bytes} [Tue Aug 2 09:05:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:05:35.015420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27775be4-a7e4-48f2-ac7e-efa39d00f5da None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:05:35.016961 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-27775be4-a7e4-48f2-ac7e-efa39d00f5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:35.146892 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27775be4-a7e4-48f2-ac7e-efa39d00f5da None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:05:35.232177 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7ca14-cc86-41f0-adfb-dfdf7c991f83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:05:35.232809 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7ca14-cc86-41f0-adfb-dfdf7c991f83 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:05:35.233095 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7ca14-cc86-41f0-adfb-dfdf7c991f83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:05:35.233407 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7ca14-cc86-41f0-adfb-dfdf7c991f83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:05:35.235074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f8d7ca14-cc86-41f0-adfb-dfdf7c991f83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:05:35.245726 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:05:35.359055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7ca14-cc86-41f0-adfb-dfdf7c991f83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:05:35.360519 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7ca14-cc86-41f0-adfb-dfdf7c991f83 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:05:35.384171 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 751/1493] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:05:35 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:05:44.971548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=82 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:05:45.381418 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=50 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:05.220048 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be24a5-f866-4d62-a002-951d7d64fc0a None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:05.220617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be24a5-f866-4d62-a002-951d7d64fc0a None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:05.220617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be24a5-f866-4d62-a002-951d7d64fc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:05.296434 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-b2be24a5-f866-4d62-a002-951d7d64fc0a None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:05.297653 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b2be24a5-f866-4d62-a002-951d7d64fc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:05.395299 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 743/1494] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:07:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:05.404026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e481f2-592e-4903-a142-5a366b1bcd15 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:05.404522 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e481f2-592e-4903-a142-5a366b1bcd15 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:05.404621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e481f2-592e-4903-a142-5a366b1bcd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:05.481469 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-93e481f2-592e-4903-a142-5a366b1bcd15 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:05.482616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-93e481f2-592e-4903-a142-5a366b1bcd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:05.579327 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 752/1495] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:07:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:05.584085 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be24a5-f866-4d62-a002-951d7d64fc0a None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:05.585478 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b2be24a5-f866-4d62-a002-951d7d64fc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:05.596657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be24a5-f866-4d62-a002-951d7d64fc0a None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:05.659638 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:05.694819 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f390611-fc0f-4825-b449-bb0b887d02bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:05.695438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f390611-fc0f-4825-b449-bb0b887d02bc admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:05.695716 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f390611-fc0f-4825-b449-bb0b887d02bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:05.695988 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f390611-fc0f-4825-b449-bb0b887d02bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:05.696673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f390611-fc0f-4825-b449-bb0b887d02bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:07:05.696977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f390611-fc0f-4825-b449-bb0b887d02bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:05.698041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: WARNING py.warnings [None req-7f390611-fc0f-4825-b449-bb0b887d02bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:07:05.698041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: warnings.warn(msg) Aug 02 09:07:05.698041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: Aug 02 09:07:05.698931 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f390611-fc0f-4825-b449-bb0b887d02bc admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:05.710357 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 744/1496] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:07:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:07:05.718755 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e481f2-592e-4903-a142-5a366b1bcd15 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:05.728102 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e481f2-592e-4903-a142-5a366b1bcd15 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:05.791605 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:05.826841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57412951-899e-403d-83d8-89a2292d2202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:05.827515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57412951-899e-403d-83d8-89a2292d2202 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:05.827782 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57412951-899e-403d-83d8-89a2292d2202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:05.828050 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57412951-899e-403d-83d8-89a2292d2202 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:05.828703 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57412951-899e-403d-83d8-89a2292d2202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:05.830451 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-57412951-899e-403d-83d8-89a2292d2202 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:07:05.830451 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Aug 02 09:07:05.830451 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: Aug 02 09:07:05.831629 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57412951-899e-403d-83d8-89a2292d2202 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:05.890166 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 753/1497] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:07:05 2022] POST /identity/v3/projects => generated 378 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:05.894389 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f390611-fc0f-4825-b449-bb0b887d02bc admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:05.895939 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-7f390611-fc0f-4825-b449-bb0b887d02bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:05.932325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f390611-fc0f-4825-b449-bb0b887d02bc admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:06.025305 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80f30e-cdd6-41f9-bf5a-f522cc5c9a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:06.025915 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80f30e-cdd6-41f9-bf5a-f522cc5c9a19 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:06.026190 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80f30e-cdd6-41f9-bf5a-f522cc5c9a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:06.026456 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80f30e-cdd6-41f9-bf5a-f522cc5c9a19 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:06.027104 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80f30e-cdd6-41f9-bf5a-f522cc5c9a19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:06.028108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: WARNING py.warnings [None req-2f80f30e-cdd6-41f9-bf5a-f522cc5c9a19 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:07:06.028108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: warnings.warn(msg) Aug 02 09:07:06.028108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: Aug 02 09:07:06.029320 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80f30e-cdd6-41f9-bf5a-f522cc5c9a19 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:06.072320 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 745/1498] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:07:05 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:06.076412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57412951-899e-403d-83d8-89a2292d2202 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:06.077935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-57412951-899e-403d-83d8-89a2292d2202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:06.141569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57412951-899e-403d-83d8-89a2292d2202 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:06.226775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:06.238500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f4e4a-b139-490c-a8ab-c5d8c57aa1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:06.239120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f4e4a-b139-490c-a8ab-c5d8c57aa1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:06.239400 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f4e4a-b139-490c-a8ab-c5d8c57aa1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:06.239691 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f4e4a-b139-490c-a8ab-c5d8c57aa1d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:06.240478 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f4e4a-b139-490c-a8ab-c5d8c57aa1d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:06.241688 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-0a2f4e4a-b139-490c-a8ab-c5d8c57aa1d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:07:06.241688 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Aug 02 09:07:06.241688 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: Aug 02 09:07:06.242568 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f4e4a-b139-490c-a8ab-c5d8c57aa1d6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:06.255366 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 754/1499] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:07:06 2022] GET /identity/v3/roles => generated 1427 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:06.259956 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80f30e-cdd6-41f9-bf5a-f522cc5c9a19 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:06.261508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-2f80f30e-cdd6-41f9-bf5a-f522cc5c9a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:06.273451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80f30e-cdd6-41f9-bf5a-f522cc5c9a19 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:06.298360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:06.369056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca23378-c572-4f90-92bf-9c7576116218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:06.369662 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca23378-c572-4f90-92bf-9c7576116218 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:06.369987 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca23378-c572-4f90-92bf-9c7576116218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:06.370259 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca23378-c572-4f90-92bf-9c7576116218 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:06.370992 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca23378-c572-4f90-92bf-9c7576116218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:06.372453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: WARNING py.warnings [None req-eca23378-c572-4f90-92bf-9c7576116218 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:07:06.372453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: warnings.warn(msg) Aug 02 09:07:06.372453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: Aug 02 09:07:06.373379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca23378-c572-4f90-92bf-9c7576116218 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:06.387425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 746/1500] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:07:06 2022] GET /identity/v3/roles => generated 1427 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:06.391873 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f4e4a-b139-490c-a8ab-c5d8c57aa1d6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:06.393437 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0a2f4e4a-b139-490c-a8ab-c5d8c57aa1d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:06.427267 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f4e4a-b139-490c-a8ab-c5d8c57aa1d6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:06.527191 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18e19c-faf2-49d0-a30b-dbecd0ac3e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:06.527858 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18e19c-faf2-49d0-a30b-dbecd0ac3e17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:06.528143 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18e19c-faf2-49d0-a30b-dbecd0ac3e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:06.528421 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18e19c-faf2-49d0-a30b-dbecd0ac3e17 admin admin] PATH_INFO: `/v3/projects/63f8b6cbf3634de399022d98108385fd/users/5da81ec80a9e449591404d7f6ff1aff0/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:06.550502 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18e19c-faf2-49d0-a30b-dbecd0ac3e17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63f8b6cbf3634de399022d98108385fd, user_id=5da81ec80a9e449591404d7f6ff1aff0, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:06.551621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-4d18e19c-faf2-49d0-a30b-dbecd0ac3e17 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:07:06.551621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Aug 02 09:07:06.551621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: Aug 02 09:07:06.554241 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18e19c-faf2-49d0-a30b-dbecd0ac3e17 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:06.581751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 755/1501] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:07:06 2022] PUT /identity/v3/projects/63f8b6cbf3634de399022d98108385fd/users/5da81ec80a9e449591404d7f6ff1aff0/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:06.594397 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a08589-0b98-4359-9fa1-c572e21a521f None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:06.594689 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a08589-0b98-4359-9fa1-c572e21a521f None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:06.594969 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a08589-0b98-4359-9fa1-c572e21a521f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:06.655033 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-b0a08589-0b98-4359-9fa1-c572e21a521f None None] MFA Rules not processed for user `5da81ec80a9e449591404d7f6ff1aff0`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:06.656165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b0a08589-0b98-4359-9fa1-c572e21a521f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:06.743042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 747/1502] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:07:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:06.756748 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8788e3-df93-4f68-ab70-5d73e7d628ea None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:06.757037 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8788e3-df93-4f68-ab70-5d73e7d628ea None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:06.757358 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8788e3-df93-4f68-ab70-5d73e7d628ea None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:06.758714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 756/1503] 10.222.0.100 () {54 vars in 969 bytes} [Tue Aug 2 09:07:06 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:07:06.764377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f084c54-181b-42f3-863e-7e9c0be9c1eb None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:06.764673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f084c54-181b-42f3-863e-7e9c0be9c1eb None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:06.764981 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f084c54-181b-42f3-863e-7e9c0be9c1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:06.796498 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:06.841348 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-9f084c54-181b-42f3-863e-7e9c0be9c1eb None None] MFA Rules not processed for user `8e55bb6b624c45a3a08306403570177d`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:06.842567 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-9f084c54-181b-42f3-863e-7e9c0be9c1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:06.919702 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 748/1504] 10.222.0.100 () {58 vars in 1066 bytes} [Tue Aug 2 09:07:06 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:07:06.927808 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8788e3-df93-4f68-ab70-5d73e7d628ea None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:06.931595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-aa8788e3-df93-4f68-ab70-5d73e7d628ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:06.984740 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:07.046931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8788e3-df93-4f68-ab70-5d73e7d628ea None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:07.125320 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5a8cf-423e-4c7a-a326-82434965bfe7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:07.126212 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a8cf-423e-4c7a-a326-82434965bfe7 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:07.126498 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a8cf-423e-4c7a-a326-82434965bfe7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:07.126776 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a8cf-423e-4c7a-a326-82434965bfe7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:07.128401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f5b5a8cf-423e-4c7a-a326-82434965bfe7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:07.233929 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5a8cf-423e-4c7a-a326-82434965bfe7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:07.235265 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5a8cf-423e-4c7a-a326-82434965bfe7 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:07.238883 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:07.254243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 757/1505] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:07:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:07.300583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f2009-2e13-45bb-8e4d-781e3ee4160e None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:07.301016 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f2009-2e13-45bb-8e4d-781e3ee4160e None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:07.301157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f2009-2e13-45bb-8e4d-781e3ee4160e None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:07.302439 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 749/1506] 10.222.0.100 () {52 vars in 835 bytes} [Tue Aug 2 09:07:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:07:07.306193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5a8cf-423e-4c7a-a326-82434965bfe7 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:07.307390 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f5b5a8cf-423e-4c7a-a326-82434965bfe7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:07.408224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5a8cf-423e-4c7a-a326-82434965bfe7 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:07.502004 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c88f52-dd53-4752-827a-1b60eadfd310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:07.502677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c88f52-dd53-4752-827a-1b60eadfd310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:07.502827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c88f52-dd53-4752-827a-1b60eadfd310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:07.503095 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c88f52-dd53-4752-827a-1b60eadfd310 admin admin] PATH_INFO: `/v3/users/5da81ec80a9e449591404d7f6ff1aff0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:07.516516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c88f52-dd53-4752-827a-1b60eadfd310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5da81ec80a9e449591404d7f6ff1aff0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:07.517651 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-02c88f52-dd53-4752-827a-1b60eadfd310 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:07:07.517651 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Aug 02 09:07:07.517651 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: Aug 02 09:07:07.518852 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c88f52-dd53-4752-827a-1b60eadfd310 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:07.529372 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:07.590862 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-02c88f52-dd53-4752-827a-1b60eadfd310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5da81ec80a9e449591404d7f6ff1aff0'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:07.605384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-02c88f52-dd53-4752-827a-1b60eadfd310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5da81ec80a9e449591404d7f6ff1aff0'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:07.605967 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-02c88f52-dd53-4752-827a-1b60eadfd310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5da81ec80a9e449591404d7f6ff1aff0'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:07.614662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-02c88f52-dd53-4752-827a-1b60eadfd310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5da81ec80a9e449591404d7f6ff1aff0'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:07.615227 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-02c88f52-dd53-4752-827a-1b60eadfd310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5da81ec80a9e449591404d7f6ff1aff0'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:07.644311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 758/1507] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:07:07 2022] DELETE /identity/v3/users/5da81ec80a9e449591404d7f6ff1aff0 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:07:07.649452 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea82b1c-96f7-4f1f-ab43-cd97e0ef0e4b None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:07.649760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea82b1c-96f7-4f1f-ab43-cd97e0ef0e4b None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:07.650064 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea82b1c-96f7-4f1f-ab43-cd97e0ef0e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:07.707709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-9ea82b1c-96f7-4f1f-ab43-cd97e0ef0e4b None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:07.708912 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-9ea82b1c-96f7-4f1f-ab43-cd97e0ef0e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:07.786130 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:07.799833 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 750/1508] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:07:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:07.810726 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c88f52-dd53-4752-827a-1b60eadfd310 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:07.811978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-02c88f52-dd53-4752-827a-1b60eadfd310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:07.916054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c88f52-dd53-4752-827a-1b60eadfd310 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:07.972869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:07.995590 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ffa2c-dc43-4138-80ac-e266968e2a51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:07.996221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ffa2c-dc43-4138-80ac-e266968e2a51 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:07.996506 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ffa2c-dc43-4138-80ac-e266968e2a51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:07.996784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ffa2c-dc43-4138-80ac-e266968e2a51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:07.998599 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c78ffa2c-dc43-4138-80ac-e266968e2a51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:08.111351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78ffa2c-dc43-4138-80ac-e266968e2a51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:08.112697 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78ffa2c-dc43-4138-80ac-e266968e2a51 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:08.132644 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 759/1509] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:07:07 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:08.511878 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea82b1c-96f7-4f1f-ab43-cd97e0ef0e4b None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:08.513181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-9ea82b1c-96f7-4f1f-ab43-cd97e0ef0e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:08.525752 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea82b1c-96f7-4f1f-ab43-cd97e0ef0e4b None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:08.623547 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3686fa-e706-4b39-a9bc-298b12e5c955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:08.624197 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3686fa-e706-4b39-a9bc-298b12e5c955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:08.624536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3686fa-e706-4b39-a9bc-298b12e5c955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:08.624817 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3686fa-e706-4b39-a9bc-298b12e5c955 admin admin] PATH_INFO: `/v3/projects/63f8b6cbf3634de399022d98108385fd` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:08.637607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3686fa-e706-4b39-a9bc-298b12e5c955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63f8b6cbf3634de399022d98108385fd)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:08.638591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: WARNING py.warnings [None req-db3686fa-e706-4b39-a9bc-298b12e5c955 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:07:08.638591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: warnings.warn(msg) Aug 02 09:07:08.638591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: Aug 02 09:07:08.639769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3686fa-e706-4b39-a9bc-298b12e5c955 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:08.697893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-db3686fa-e706-4b39-a9bc-298b12e5c955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63f8b6cbf3634de399022d98108385fd', 'request_id': 'req-db3686fa-e706-4b39-a9bc-298b12e5c955'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:08.711474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-db3686fa-e706-4b39-a9bc-298b12e5c955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63f8b6cbf3634de399022d98108385fd', 'request_id': 'req-db3686fa-e706-4b39-a9bc-298b12e5c955'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:08.722096 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-db3686fa-e706-4b39-a9bc-298b12e5c955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63f8b6cbf3634de399022d98108385fd', 'request_id': 'req-db3686fa-e706-4b39-a9bc-298b12e5c955'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:08.747811 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-db3686fa-e706-4b39-a9bc-298b12e5c955 admin admin] The token cache is being invalidate because project 63f8b6cbf3634de399022d98108385fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:08.748106 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-db3686fa-e706-4b39-a9bc-298b12e5c955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:08.748536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-db3686fa-e706-4b39-a9bc-298b12e5c955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:08.750101 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 751/1510] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:07:08 2022] DELETE /identity/v3/projects/63f8b6cbf3634de399022d98108385fd => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:18.129723 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=62 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:18.745805 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:52.776443 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171adf1-2412-428d-99d5-01a798136657 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:52.777049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171adf1-2412-428d-99d5-01a798136657 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:52.777049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171adf1-2412-428d-99d5-01a798136657 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:52.778407 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:08:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:08:52.783623 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9ef0b-33a5-467a-bb88-c268edc2d0b4 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:52.784178 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9ef0b-33a5-467a-bb88-c268edc2d0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:52.784285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9ef0b-33a5-467a-bb88-c268edc2d0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:52.871471 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-a2e9ef0b-33a5-467a-bb88-c268edc2d0b4 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:52.872702 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-a2e9ef0b-33a5-467a-bb88-c268edc2d0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:52.981149 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:08:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:08:53.144597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171adf1-2412-428d-99d5-01a798136657 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:53.147461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a171adf1-2412-428d-99d5-01a798136657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:53.271005 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171adf1-2412-428d-99d5-01a798136657 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:53.277537 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d3f35-82ce-4d1e-a6a1-3ff373712883 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:53.278160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d3f35-82ce-4d1e-a6a1-3ff373712883 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:53.278454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d3f35-82ce-4d1e-a6a1-3ff373712883 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:53.278731 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d3f35-82ce-4d1e-a6a1-3ff373712883 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:53.287014 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d3f35-82ce-4d1e-a6a1-3ff373712883 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:53.288624 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d3f35-82ce-4d1e-a6a1-3ff373712883 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:53.298140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 761/1513] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:08:53 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:08:54.142422 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7894ce-7017-4d9b-97c2-2fbf8aac9d57 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:54.143020 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7894ce-7017-4d9b-97c2-2fbf8aac9d57 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:54.143020 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7894ce-7017-4d9b-97c2-2fbf8aac9d57 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:54.144434 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 753/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:08:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:08:54.149589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee5780-b6f9-41d7-b0be-1cbcc4557042 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:54.150065 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee5780-b6f9-41d7-b0be-1cbcc4557042 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:54.150184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee5780-b6f9-41d7-b0be-1cbcc4557042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:54.171502 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ebee5780-b6f9-41d7-b0be-1cbcc4557042 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:54.172636 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ebee5780-b6f9-41d7-b0be-1cbcc4557042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:54.184962 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 762/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:08:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:08:54.337302 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7894ce-7017-4d9b-97c2-2fbf8aac9d57 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:54.351654 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7894ce-7017-4d9b-97c2-2fbf8aac9d57 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:54.357424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a2162-4ad8-4816-9b9f-4d2b6c5a967d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:54.358041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a2162-4ad8-4816-9b9f-4d2b6c5a967d service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:54.358325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a2162-4ad8-4816-9b9f-4d2b6c5a967d service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:54.358600 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a2162-4ad8-4816-9b9f-4d2b6c5a967d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:54.368750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a2162-4ad8-4816-9b9f-4d2b6c5a967d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:54.370470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a2162-4ad8-4816-9b9f-4d2b6c5a967d service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:54.380860 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 754/1516] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:08:54 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:08:54.964837 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d79c3-a7ec-4dc5-ad86-a74f727a4a6a None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:54.965162 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d79c3-a7ec-4dc5-ad86-a74f727a4a6a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:54.965446 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d79c3-a7ec-4dc5-ad86-a74f727a4a6a None None] PATH_INFO: `/v3` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:54.966765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 763/1517] 10.222.0.70 () {54 vars in 929 bytes} [Tue Aug 2 09:08:54 2022] GET /identity/v3 => generated 256 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:08:54.976320 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca954b5c-b5f8-40ea-a2fd-ef9dd4009bd0 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:54.976606 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca954b5c-b5f8-40ea-a2fd-ef9dd4009bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:54.976889 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca954b5c-b5f8-40ea-a2fd-ef9dd4009bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:55.039181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-ca954b5c-b5f8-40ea-a2fd-ef9dd4009bd0 None None] MFA Rules not processed for user `b40981fe179a43dc81553d3594dad649`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:55.040404 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ca954b5c-b5f8-40ea-a2fd-ef9dd4009bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:55.057966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:55.116300 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 755/1518] 10.222.0.70 () {58 vars in 1019 bytes} [Tue Aug 2 09:08:54 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:04.166479 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=56 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:05.112792 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=23 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:05.645683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3d8f6-908e-42b6-829b-23619b91138d None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:05.646294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3d8f6-908e-42b6-829b-23619b91138d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:05.646294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3d8f6-908e-42b6-829b-23619b91138d None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:05.647572 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 764/1519] 10.222.0.100 () {54 vars in 966 bytes} [Tue Aug 2 09:09:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:09:05.652824 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4df0b-9e0b-4efb-8f5c-0a29abfaa458 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:05.653168 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4df0b-9e0b-4efb-8f5c-0a29abfaa458 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:05.653477 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4df0b-9e0b-4efb-8f5c-0a29abfaa458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:05.661167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee343521-4501-4dec-870d-724bdd6de047 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:05.661167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee343521-4501-4dec-870d-724bdd6de047 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:05.661167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee343521-4501-4dec-870d-724bdd6de047 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:05.663100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 765/1520] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:09:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 09:09:05.670975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d805ab-08be-4539-9a47-aaaf4e9b2efb None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:05.671395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d805ab-08be-4539-9a47-aaaf4e9b2efb None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:05.671730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d805ab-08be-4539-9a47-aaaf4e9b2efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:05.675283 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-89f4df0b-9e0b-4efb-8f5c-0a29abfaa458 None None] MFA Rules not processed for user `b40981fe179a43dc81553d3594dad649`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:05.676520 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-89f4df0b-9e0b-4efb-8f5c-0a29abfaa458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:05.687055 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 756/1521] 10.222.0.100 () {58 vars in 1063 bytes} [Tue Aug 2 09:09:05 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:05.693736 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-66d805ab-08be-4539-9a47-aaaf4e9b2efb None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:05.694963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-66d805ab-08be-4539-9a47-aaaf4e9b2efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:05.695184 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4df0b-9e0b-4efb-8f5c-0a29abfaa458 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:05.703005 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 766/1522] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:09:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:09:05.705081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4df0b-9e0b-4efb-8f5c-0a29abfaa458 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:05.710676 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79cfe3-58ce-41d5-b7c7-35bc1de965ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b40981fe179a43dc81553d3594dad649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:05.711297 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79cfe3-58ce-41d5-b7c7-35bc1de965ce service placement] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:05.711596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79cfe3-58ce-41d5-b7c7-35bc1de965ce service placement] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:05.711874 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79cfe3-58ce-41d5-b7c7-35bc1de965ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:05.720845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c79cfe3-58ce-41d5-b7c7-35bc1de965ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:05.722467 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c79cfe3-58ce-41d5-b7c7-35bc1de965ce service placement] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:05.732983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 757/1523] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:09:05 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:09:05.868617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d805ab-08be-4539-9a47-aaaf4e9b2efb None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:05.882160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d805ab-08be-4539-9a47-aaaf4e9b2efb None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:05.887852 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25a143-37d0-4f7c-891b-5e563a466257 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:05.888581 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25a143-37d0-4f7c-891b-5e563a466257 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:05.888936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25a143-37d0-4f7c-891b-5e563a466257 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:05.889315 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25a143-37d0-4f7c-891b-5e563a466257 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:05.900878 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25a143-37d0-4f7c-891b-5e563a466257 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:05.902446 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25a143-37d0-4f7c-891b-5e563a466257 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:05.913750 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 767/1524] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:09:05 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:09:06.783442 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5193c-41d0-4af2-a53a-97960bef43c6 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:06.783998 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5193c-41d0-4af2-a53a-97960bef43c6 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:06.784125 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5193c-41d0-4af2-a53a-97960bef43c6 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:06.786336 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:09:06 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:09:06.791538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f599c0-60fb-44a5-afdc-c2b84fcc1913 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:06.792019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f599c0-60fb-44a5-afdc-c2b84fcc1913 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:06.792143 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f599c0-60fb-44a5-afdc-c2b84fcc1913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:06.811794 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-04f599c0-60fb-44a5-afdc-c2b84fcc1913 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:06.813007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-04f599c0-60fb-44a5-afdc-c2b84fcc1913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:06.821570 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:09:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:06.975074 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5193c-41d0-4af2-a53a-97960bef43c6 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:06.991825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5193c-41d0-4af2-a53a-97960bef43c6 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:06.997285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763315b-f376-4f16-b101-268e9232ede8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:06.997905 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763315b-f376-4f16-b101-268e9232ede8 service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:06.998191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763315b-f376-4f16-b101-268e9232ede8 service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:06.998469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763315b-f376-4f16-b101-268e9232ede8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:07.008069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763315b-f376-4f16-b101-268e9232ede8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:07.009713 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763315b-f376-4f16-b101-268e9232ede8 service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:07.019438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 759/1527] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:09:06 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:09:11.150904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb9b5e-1596-4a49-9ef1-1f1d4f686a63 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:11.151497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb9b5e-1596-4a49-9ef1-1f1d4f686a63 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:11.151497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb9b5e-1596-4a49-9ef1-1f1d4f686a63 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:11.152888 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:09:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:09:11.158376 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5a4fb-e4cd-4d47-84d5-ad2ca7a8024a None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:11.158860 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5a4fb-e4cd-4d47-84d5-ad2ca7a8024a None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:11.158973 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5a4fb-e4cd-4d47-84d5-ad2ca7a8024a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:11.179876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-66e5a4fb-e4cd-4d47-84d5-ad2ca7a8024a None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:11.180998 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-66e5a4fb-e4cd-4d47-84d5-ad2ca7a8024a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:11.188899 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:09:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:11.268477 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb9b5e-1596-4a49-9ef1-1f1d4f686a63 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:11.270449 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b2fb9b5e-1596-4a49-9ef1-1f1d4f686a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:11.357843 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb9b5e-1596-4a49-9ef1-1f1d4f686a63 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:11.361623 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba18218-6872-4469-ac93-7be5ee783b31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:11.362254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba18218-6872-4469-ac93-7be5ee783b31 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:11.362537 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba18218-6872-4469-ac93-7be5ee783b31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:11.362814 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba18218-6872-4469-ac93-7be5ee783b31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:11.374250 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba18218-6872-4469-ac93-7be5ee783b31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:11.375686 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba18218-6872-4469-ac93-7be5ee783b31 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:11.386094 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 770/1530] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:09:11 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:09:13.441986 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bce7b-da3b-4394-9800-db1999235c80 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:13.442564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bce7b-da3b-4394-9800-db1999235c80 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:13.442564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bce7b-da3b-4394-9800-db1999235c80 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:13.444472 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:09:13 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:09:13.450020 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7e149-5e02-4dd9-8eb0-3af9d7bfd272 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:13.450772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7e149-5e02-4dd9-8eb0-3af9d7bfd272 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:13.450772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7e149-5e02-4dd9-8eb0-3af9d7bfd272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:13.470729 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1ca7e149-5e02-4dd9-8eb0-3af9d7bfd272 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:13.471944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1ca7e149-5e02-4dd9-8eb0-3af9d7bfd272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:13.482684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:09:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:13.520892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368bce7b-da3b-4394-9800-db1999235c80 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:13.523845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-368bce7b-da3b-4394-9800-db1999235c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:13.536816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368bce7b-da3b-4394-9800-db1999235c80 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:13.541218 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0bc324-7a58-4bfd-9b88-bcfd9b71c5a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:13.541833 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bc324-7a58-4bfd-9b88-bcfd9b71c5a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:13.542115 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bc324-7a58-4bfd-9b88-bcfd9b71c5a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:13.542388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bc324-7a58-4bfd-9b88-bcfd9b71c5a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:13.552146 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0bc324-7a58-4bfd-9b88-bcfd9b71c5a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:13.554212 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0bc324-7a58-4bfd-9b88-bcfd9b71c5a3 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:13.564070 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 762/1533] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:09:13 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:09:17.257875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a6844-1e89-4f0e-87a9-a5b8fc28ba8a None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:17.258409 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a6844-1e89-4f0e-87a9-a5b8fc28ba8a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:17.258409 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a6844-1e89-4f0e-87a9-a5b8fc28ba8a None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:17.259885 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:09:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:09:17.265472 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db388b-36ec-4c94-bdb8-83b924775be0 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:17.266867 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db388b-36ec-4c94-bdb8-83b924775be0 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:17.266867 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db388b-36ec-4c94-bdb8-83b924775be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:17.285862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-22db388b-36ec-4c94-bdb8-83b924775be0 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:17.287094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-22db388b-36ec-4c94-bdb8-83b924775be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:17.294924 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:09:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:09:17.607414 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a6844-1e89-4f0e-87a9-a5b8fc28ba8a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:17.609168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a67a6844-1e89-4f0e-87a9-a5b8fc28ba8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:17.693515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a6844-1e89-4f0e-87a9-a5b8fc28ba8a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:17.698274 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780ad09-20be-4b40-85e3-9188414abec5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:17.698916 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780ad09-20be-4b40-85e3-9188414abec5 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:17.699203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780ad09-20be-4b40-85e3-9188414abec5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:17.699482 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780ad09-20be-4b40-85e3-9188414abec5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:17.709613 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780ad09-20be-4b40-85e3-9188414abec5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:17.711053 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780ad09-20be-4b40-85e3-9188414abec5 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:17.721058 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 773/1536] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:09:17 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:09:19.825334 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b25e2-295b-45ae-a3a2-c1b2ae289f4a None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:19.825931 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b25e2-295b-45ae-a3a2-c1b2ae289f4a None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:19.825931 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b25e2-295b-45ae-a3a2-c1b2ae289f4a None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:19.827254 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:09:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:09:19.832726 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccc6f2-da28-4721-b194-eb2c9c3ea93b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:19.833221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccc6f2-da28-4721-b194-eb2c9c3ea93b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:19.833319 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccc6f2-da28-4721-b194-eb2c9c3ea93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:19.857149 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-abccc6f2-da28-4721-b194-eb2c9c3ea93b None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:19.858810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-abccc6f2-da28-4721-b194-eb2c9c3ea93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:19.867903 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:09:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:20.177419 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b25e2-295b-45ae-a3a2-c1b2ae289f4a None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:20.179185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ca4b25e2-295b-45ae-a3a2-c1b2ae289f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:20.192317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b25e2-295b-45ae-a3a2-c1b2ae289f4a None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:20.196470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d010aa-1762-4150-8607-c35e5501e393 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:20.197089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d010aa-1762-4150-8607-c35e5501e393 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:20.197408 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d010aa-1762-4150-8607-c35e5501e393 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:20.197690 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d010aa-1762-4150-8607-c35e5501e393 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:20.214685 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d010aa-1762-4150-8607-c35e5501e393 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:20.217020 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d010aa-1762-4150-8607-c35e5501e393 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:20.227544 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 765/1539] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:09:20 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:09:29.848397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=88 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:30.222892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=33 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:49.711354 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0327f77-e68e-4ad8-ac01-38835e2bbd2d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:49.712365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0327f77-e68e-4ad8-ac01-38835e2bbd2d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:49.712567 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0327f77-e68e-4ad8-ac01-38835e2bbd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:49.748461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c0327f77-e68e-4ad8-ac01-38835e2bbd2d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:49.749891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c0327f77-e68e-4ad8-ac01-38835e2bbd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:49.770983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 775/1540] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:10:49.781590 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d787a3-410f-432a-9982-aacc4ef12c97 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:49.782170 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d787a3-410f-432a-9982-aacc4ef12c97 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:49.782170 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d787a3-410f-432a-9982-aacc4ef12c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:49.813565 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-41d787a3-410f-432a-9982-aacc4ef12c97 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:49.814764 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-41d787a3-410f-432a-9982-aacc4ef12c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:49.826364 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 766/1541] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:49.831560 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0327f77-e68e-4ad8-ac01-38835e2bbd2d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:49.845839 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0327f77-e68e-4ad8-ac01-38835e2bbd2d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:49.852836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd38ae-954f-4d03-bd7b-14af6d8d8065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:49.853511 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd38ae-954f-4d03-bd7b-14af6d8d8065 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:49.853793 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd38ae-954f-4d03-bd7b-14af6d8d8065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:49.854069 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd38ae-954f-4d03-bd7b-14af6d8d8065 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:49.854785 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd38ae-954f-4d03-bd7b-14af6d8d8065 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:10:49.855092 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd38ae-954f-4d03-bd7b-14af6d8d8065 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:49.856121 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-2ccd38ae-954f-4d03-bd7b-14af6d8d8065 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:10:49.856121 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Aug 02 09:10:49.856121 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: Aug 02 09:10:49.857172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd38ae-954f-4d03-bd7b-14af6d8d8065 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:49.878815 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 776/1542] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:10:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:10:49.888301 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d787a3-410f-432a-9982-aacc4ef12c97 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:49.904221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d787a3-410f-432a-9982-aacc4ef12c97 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:49.909252 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f0c90-5071-45d6-a128-46ae7c70a966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:49.910076 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f0c90-5071-45d6-a128-46ae7c70a966 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:49.910935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f0c90-5071-45d6-a128-46ae7c70a966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:49.911313 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f0c90-5071-45d6-a128-46ae7c70a966 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:49.912367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f0c90-5071-45d6-a128-46ae7c70a966 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:49.919384 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: WARNING py.warnings [None req-fc8f0c90-5071-45d6-a128-46ae7c70a966 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:10:49.919384 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: warnings.warn(msg) Aug 02 09:10:49.919384 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: Aug 02 09:10:49.920642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f0c90-5071-45d6-a128-46ae7c70a966 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:49.949931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15eb814-9d28-44ab-97b2-0d30497ccb80 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:49.950284 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15eb814-9d28-44ab-97b2-0d30497ccb80 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:49.950514 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15eb814-9d28-44ab-97b2-0d30497ccb80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:49.998703 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b15eb814-9d28-44ab-97b2-0d30497ccb80 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:50.000169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b15eb814-9d28-44ab-97b2-0d30497ccb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:50.015100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 777/1543] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:50.016293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633fd75-67d2-4e4b-a637-b98ab18a5004 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.016694 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633fd75-67d2-4e4b-a637-b98ab18a5004 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.016915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633fd75-67d2-4e4b-a637-b98ab18a5004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.025476 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 767/1544] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:10:49 2022] POST /identity/v3/projects => generated 380 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:10:50.027631 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2867c-51b7-46a2-a2fe-c22818523bbf None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.028198 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2867c-51b7-46a2-a2fe-c22818523bbf None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.028691 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2867c-51b7-46a2-a2fe-c22818523bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.049294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6633fd75-67d2-4e4b-a637-b98ab18a5004 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:50.050899 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6633fd75-67d2-4e4b-a637-b98ab18a5004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:50.051409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-62b2867c-51b7-46a2-a2fe-c22818523bbf None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:50.052855 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-62b2867c-51b7-46a2-a2fe-c22818523bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:50.118004 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 778/1545] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:50.119624 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7436bf-de4e-4bd8-8b88-f8eba8ac1db7 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.120060 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7436bf-de4e-4bd8-8b88-f8eba8ac1db7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.120347 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7436bf-de4e-4bd8-8b88-f8eba8ac1db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.125997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 768/1546] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:50.125997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7798df-4d7c-40c2-acc4-b8bfa69377ac None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.125997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7798df-4d7c-40c2-acc4-b8bfa69377ac None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.125997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7798df-4d7c-40c2-acc4-b8bfa69377ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.146871 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-fa7436bf-de4e-4bd8-8b88-f8eba8ac1db7 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:50.148113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fa7436bf-de4e-4bd8-8b88-f8eba8ac1db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:50.149527 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-8c7798df-4d7c-40c2-acc4-b8bfa69377ac None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:50.151135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-8c7798df-4d7c-40c2-acc4-b8bfa69377ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:50.156657 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 779/1547] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:50.157552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7436bf-de4e-4bd8-8b88-f8eba8ac1db7 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:50.165426 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 769/1548] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:50.167211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32b3b6-b46d-4200-89b7-db7dabfc34a9 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.167566 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32b3b6-b46d-4200-89b7-db7dabfc34a9 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.167912 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32b3b6-b46d-4200-89b7-db7dabfc34a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.171526 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7436bf-de4e-4bd8-8b88-f8eba8ac1db7 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:50.176875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc45779-da89-446b-b328-7aae182b83ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:50.177564 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc45779-da89-446b-b328-7aae182b83ff admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.177849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc45779-da89-446b-b328-7aae182b83ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.178128 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc45779-da89-446b-b328-7aae182b83ff admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.178826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc45779-da89-446b-b328-7aae182b83ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:50.179869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-1dc45779-da89-446b-b328-7aae182b83ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:10:50.179869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Aug 02 09:10:50.179869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: Aug 02 09:10:50.181060 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc45779-da89-446b-b328-7aae182b83ff admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:50.188668 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-df32b3b6-b46d-4200-89b7-db7dabfc34a9 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:50.189930 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-df32b3b6-b46d-4200-89b7-db7dabfc34a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:50.199669 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 770/1549] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:50.200670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87749d-b4cd-4ed1-8256-63c8b367047d None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.200958 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87749d-b4cd-4ed1-8256-63c8b367047d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.201270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87749d-b4cd-4ed1-8256-63c8b367047d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.208188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 780/1550] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/users => generated 291 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:10:50.208188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bba55-120b-4577-b982-a86c7b9f25d7 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.208188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bba55-120b-4577-b982-a86c7b9f25d7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.208188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bba55-120b-4577-b982-a86c7b9f25d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.224441 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-6b87749d-b4cd-4ed1-8256-63c8b367047d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:50.225693 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-6b87749d-b4cd-4ed1-8256-63c8b367047d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:50.226560 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-7c3bba55-120b-4577-b982-a86c7b9f25d7 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:50.227813 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7c3bba55-120b-4577-b982-a86c7b9f25d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:50.233616 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 771/1551] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:50.234811 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0944705-9207-4f8f-aef4-52e2e4794541 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.235099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0944705-9207-4f8f-aef4-52e2e4794541 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.235377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0944705-9207-4f8f-aef4-52e2e4794541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.236203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 781/1552] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:50.237869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1843b-764d-4a09-b547-c50422ea5f14 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.238275 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1843b-764d-4a09-b547-c50422ea5f14 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.239515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1843b-764d-4a09-b547-c50422ea5f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.261149 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-2fa1843b-764d-4a09-b547-c50422ea5f14 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:50.262355 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2fa1843b-764d-4a09-b547-c50422ea5f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:50.263583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-f0944705-9207-4f8f-aef4-52e2e4794541 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:50.264752 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f0944705-9207-4f8f-aef4-52e2e4794541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:50.270513 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 782/1553] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:50.274333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074bbac-bd3b-4d51-a556-da1823ae2cf2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.274333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074bbac-bd3b-4d51-a556-da1823ae2cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.274333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074bbac-bd3b-4d51-a556-da1823ae2cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.274606 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 772/1554] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:50.275060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc21408-5ed1-49ba-93d5-6259eaadbfd0 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.275356 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc21408-5ed1-49ba-93d5-6259eaadbfd0 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.275699 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc21408-5ed1-49ba-93d5-6259eaadbfd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.293849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e074bbac-bd3b-4d51-a556-da1823ae2cf2 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:50.295070 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e074bbac-bd3b-4d51-a556-da1823ae2cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:50.298053 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-5cc21408-5ed1-49ba-93d5-6259eaadbfd0 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:50.299274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-5cc21408-5ed1-49ba-93d5-6259eaadbfd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:50.303676 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 783/1555] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:50.304923 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9e2da-5d72-44f8-85bf-c0fd36ea8eec None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.305309 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9e2da-5d72-44f8-85bf-c0fd36ea8eec None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.305592 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9e2da-5d72-44f8-85bf-c0fd36ea8eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.313026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 773/1556] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:50.313991 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc21408-5ed1-49ba-93d5-6259eaadbfd0 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:50.326051 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc21408-5ed1-49ba-93d5-6259eaadbfd0 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:50.327775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-85e9e2da-5d72-44f8-85bf-c0fd36ea8eec None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:50.328984 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-85e9e2da-5d72-44f8-85bf-c0fd36ea8eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:50.331587 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951abbc-d745-4190-aedb-07c195352f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:50.332358 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951abbc-d745-4190-aedb-07c195352f17 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.332645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951abbc-d745-4190-aedb-07c195352f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.332918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951abbc-d745-4190-aedb-07c195352f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.333658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951abbc-d745-4190-aedb-07c195352f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:50.335291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951abbc-d745-4190-aedb-07c195352f17 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:50.337501 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 784/1557] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:50.338115 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9e2da-5d72-44f8-85bf-c0fd36ea8eec None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:50.349150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 774/1558] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:10:50 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:10:50.349150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9013f-c0c2-412f-b71d-9183cd8c04fc None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.349150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9013f-c0c2-412f-b71d-9183cd8c04fc None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.349150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9013f-c0c2-412f-b71d-9183cd8c04fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.350489 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9e2da-5d72-44f8-85bf-c0fd36ea8eec None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:50.355184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abcceb-54d1-44e6-9aed-669379df470b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:50.355877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abcceb-54d1-44e6-9aed-669379df470b admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.356183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abcceb-54d1-44e6-9aed-669379df470b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.356495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abcceb-54d1-44e6-9aed-669379df470b admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.357237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97abcceb-54d1-44e6-9aed-669379df470b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:10:50.357547 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97abcceb-54d1-44e6-9aed-669379df470b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:50.359139 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97abcceb-54d1-44e6-9aed-669379df470b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:50.368312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-53e9013f-c0c2-412f-b71d-9183cd8c04fc None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:50.371245 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-53e9013f-c0c2-412f-b71d-9183cd8c04fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:50.373102 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 785/1559] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:10:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:10:50.373924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abcceb-54d1-44e6-9aed-669379df470b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:50.385017 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abcceb-54d1-44e6-9aed-669379df470b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:50.385832 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 775/1560] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:50.386686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9013f-c0c2-412f-b71d-9183cd8c04fc None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:50.393946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70018aea-7964-420f-b4ba-93546256a8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:50.394633 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70018aea-7964-420f-b4ba-93546256a8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.395000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70018aea-7964-420f-b4ba-93546256a8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.395229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70018aea-7964-420f-b4ba-93546256a8f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.395923 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70018aea-7964-420f-b4ba-93546256a8f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:10:50.396229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70018aea-7964-420f-b4ba-93546256a8f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:50.396783 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9013f-c0c2-412f-b71d-9183cd8c04fc None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:50.397897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70018aea-7964-420f-b4ba-93546256a8f7 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:50.406180 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d344186-8b0c-4809-9edd-8ebbbfdbd501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:50.406806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d344186-8b0c-4809-9edd-8ebbbfdbd501 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.407089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d344186-8b0c-4809-9edd-8ebbbfdbd501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.407359 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d344186-8b0c-4809-9edd-8ebbbfdbd501 admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.408057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d344186-8b0c-4809-9edd-8ebbbfdbd501 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:10:50.408364 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d344186-8b0c-4809-9edd-8ebbbfdbd501 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:50.410006 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d344186-8b0c-4809-9edd-8ebbbfdbd501 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:50.415517 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 786/1561] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:10:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:10:50.416368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70018aea-7964-420f-b4ba-93546256a8f7 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:50.422138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 776/1562] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:10:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:10:50.424076 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d344186-8b0c-4809-9edd-8ebbbfdbd501 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:50.427871 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70018aea-7964-420f-b4ba-93546256a8f7 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:50.433145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057b1d7-4719-4db0-9285-d4b58a382eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:50.433981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057b1d7-4719-4db0-9285-d4b58a382eec admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.434270 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057b1d7-4719-4db0-9285-d4b58a382eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.434548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057b1d7-4719-4db0-9285-d4b58a382eec admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.434730 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d344186-8b0c-4809-9edd-8ebbbfdbd501 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:50.435248 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057b1d7-4719-4db0-9285-d4b58a382eec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:10:50.435560 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057b1d7-4719-4db0-9285-d4b58a382eec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:50.437218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057b1d7-4719-4db0-9285-d4b58a382eec admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:50.439385 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27dcfd-9f3e-4058-b799-8d867320e69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:50.439991 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27dcfd-9f3e-4058-b799-8d867320e69c admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.440272 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27dcfd-9f3e-4058-b799-8d867320e69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.440548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27dcfd-9f3e-4058-b799-8d867320e69c admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.441267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb27dcfd-9f3e-4058-b799-8d867320e69c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:10:50.442237 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb27dcfd-9f3e-4058-b799-8d867320e69c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:50.443889 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb27dcfd-9f3e-4058-b799-8d867320e69c admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:50.450381 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 787/1563] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:10:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:10:50.451296 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057b1d7-4719-4db0-9285-d4b58a382eec admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:50.466449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 777/1564] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:10:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:10:50.466449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27dcfd-9f3e-4058-b799-8d867320e69c admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:50.469963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057b1d7-4719-4db0-9285-d4b58a382eec admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:50.471748 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27dcfd-9f3e-4058-b799-8d867320e69c admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:50.475377 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8f5a0-40eb-4d84-aacb-4335a2687c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:50.476002 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8f5a0-40eb-4d84-aacb-4335a2687c1b admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.476288 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8f5a0-40eb-4d84-aacb-4335a2687c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.476567 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8f5a0-40eb-4d84-aacb-4335a2687c1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.477080 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db363c-555c-40df-bdcc-80f2de7362a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:50.477299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8f5a0-40eb-4d84-aacb-4335a2687c1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:10:50.477608 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8f5a0-40eb-4d84-aacb-4335a2687c1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:50.477739 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db363c-555c-40df-bdcc-80f2de7362a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.478027 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db363c-555c-40df-bdcc-80f2de7362a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.478307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db363c-555c-40df-bdcc-80f2de7362a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.478995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db363c-555c-40df-bdcc-80f2de7362a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:50.479239 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8f5a0-40eb-4d84-aacb-4335a2687c1b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:50.480588 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db363c-555c-40df-bdcc-80f2de7362a7 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:50.493012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 788/1565] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:10:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:10:50.495774 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 778/1566] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:10:50 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:10:50.495898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8f5a0-40eb-4d84-aacb-4335a2687c1b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:50.495994 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db363c-555c-40df-bdcc-80f2de7362a7 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:50.505841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8f5a0-40eb-4d84-aacb-4335a2687c1b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:50.506733 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db363c-555c-40df-bdcc-80f2de7362a7 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:50.510468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcc526-aa06-4b9e-a951-fb1056dba047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:50.511085 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcc526-aa06-4b9e-a951-fb1056dba047 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.511426 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcc526-aa06-4b9e-a951-fb1056dba047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.511585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a99f5-2801-4e04-a823-ef431d491377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:50.511736 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcc526-aa06-4b9e-a951-fb1056dba047 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.512139 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a99f5-2801-4e04-a823-ef431d491377 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.512316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fcc526-aa06-4b9e-a951-fb1056dba047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:50.512466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a99f5-2801-4e04-a823-ef431d491377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.512686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a99f5-2801-4e04-a823-ef431d491377 admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.513407 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a99f5-2801-4e04-a823-ef431d491377 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:10:50.513708 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a99f5-2801-4e04-a823-ef431d491377 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:50.514225 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fcc526-aa06-4b9e-a951-fb1056dba047 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:50.515282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a99f5-2801-4e04-a823-ef431d491377 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:50.528081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 779/1567] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:10:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:10:50.529032 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a99f5-2801-4e04-a823-ef431d491377 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:50.542672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a99f5-2801-4e04-a823-ef431d491377 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:50.544495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 789/1568] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/projects => generated 390 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:10:50.545172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcc526-aa06-4b9e-a951-fb1056dba047 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:50.556827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:50.567968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:50.608075 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55707777-b817-490f-87c0-f33ab764bdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:50.608728 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55707777-b817-490f-87c0-f33ab764bdbe admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.608995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55707777-b817-490f-87c0-f33ab764bdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.609303 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55707777-b817-490f-87c0-f33ab764bdbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.609972 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55707777-b817-490f-87c0-f33ab764bdbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:50.611896 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55707777-b817-490f-87c0-f33ab764bdbe admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:50.614319 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcc526-aa06-4b9e-a951-fb1056dba047 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:50.621054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654bab3-3514-4dcb-99b0-8a6d3cf271d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:50.621725 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654bab3-3514-4dcb-99b0-8a6d3cf271d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.622018 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654bab3-3514-4dcb-99b0-8a6d3cf271d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.622292 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654bab3-3514-4dcb-99b0-8a6d3cf271d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.622975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654bab3-3514-4dcb-99b0-8a6d3cf271d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:50.624896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654bab3-3514-4dcb-99b0-8a6d3cf271d8 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:50.643601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 780/1569] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/projects => generated 404 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:10:50.644859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55707777-b817-490f-87c0-f33ab764bdbe admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:50.656969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 790/1570] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/projects => generated 384 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:10:50.657936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654bab3-3514-4dcb-99b0-8a6d3cf271d8 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:50.716681 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55707777-b817-490f-87c0-f33ab764bdbe admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:50.721474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db47483-43e1-407b-828c-8e9bcf1ca773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:50.722104 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db47483-43e1-407b-828c-8e9bcf1ca773 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.722284 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654bab3-3514-4dcb-99b0-8a6d3cf271d8 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:50.723360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db47483-43e1-407b-828c-8e9bcf1ca773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.723360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db47483-43e1-407b-828c-8e9bcf1ca773 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.723845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db47483-43e1-407b-828c-8e9bcf1ca773 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:50.725795 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db47483-43e1-407b-828c-8e9bcf1ca773 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:50.727449 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914a005-d949-4f8a-b160-a032ad65d82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:50.728066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914a005-d949-4f8a-b160-a032ad65d82a admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.728346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914a005-d949-4f8a-b160-a032ad65d82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.728618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914a005-d949-4f8a-b160-a032ad65d82a admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.729333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914a005-d949-4f8a-b160-a032ad65d82a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:50.731361 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914a005-d949-4f8a-b160-a032ad65d82a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:50.758208 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 781/1571] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/projects => generated 372 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:10:50.758208 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db47483-43e1-407b-828c-8e9bcf1ca773 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:50.772360 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 791/1572] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/projects => generated 390 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:10:50.772360 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914a005-d949-4f8a-b160-a032ad65d82a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:50.820443 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db47483-43e1-407b-828c-8e9bcf1ca773 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:50.840961 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914a005-d949-4f8a-b160-a032ad65d82a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:50.846034 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a72e96-1909-4469-966c-6094d945e224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:50.846686 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a72e96-1909-4469-966c-6094d945e224 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.846953 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a72e96-1909-4469-966c-6094d945e224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.847233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a72e96-1909-4469-966c-6094d945e224 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.848473 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a72e96-1909-4469-966c-6094d945e224 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:50.850442 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a72e96-1909-4469-966c-6094d945e224 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:50.862174 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:50.880743 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 792/1573] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/projects => generated 374 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:10:50.881775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a72e96-1909-4469-966c-6094d945e224 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:50.885655 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d89390-84c0-4e86-9650-5a11e962cedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:50.886345 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d89390-84c0-4e86-9650-5a11e962cedc admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.886630 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d89390-84c0-4e86-9650-5a11e962cedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.886910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d89390-84c0-4e86-9650-5a11e962cedc admin admin] PATH_INFO: `/v3/projects/cde77e494bb54329a58883c255a8c2c0/users/c2782f4783624ca39ace71143b3b9237/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:50.887640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:50.904515 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d89390-84c0-4e86-9650-5a11e962cedc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cde77e494bb54329a58883c255a8c2c0, user_id=c2782f4783624ca39ace71143b3b9237, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:50.906392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: WARNING py.warnings [None req-b9d89390-84c0-4e86-9650-5a11e962cedc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:10:50.906392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: warnings.warn(msg) Aug 02 09:10:50.906392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: Aug 02 09:10:50.908344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d89390-84c0-4e86-9650-5a11e962cedc admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:50.926234 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 782/1574] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:10:50 2022] PUT /identity/v3/projects/cde77e494bb54329a58883c255a8c2c0/users/c2782f4783624ca39ace71143b3b9237/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:10:50.930261 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d89390-84c0-4e86-9650-5a11e962cedc admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:50.951944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a72e96-1909-4469-966c-6094d945e224 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:50.993592 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d89390-84c0-4e86-9650-5a11e962cedc admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:50.998179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0fcb5-b456-4660-a556-f40de75ad589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:50.998802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0fcb5-b456-4660-a556-f40de75ad589 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:50.999078 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0fcb5-b456-4660-a556-f40de75ad589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:50.999354 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0fcb5-b456-4660-a556-f40de75ad589 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.000014 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e0fcb5-b456-4660-a556-f40de75ad589 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.001961 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e0fcb5-b456-4660-a556-f40de75ad589 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.011668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87203be7-1191-409a-81d4-d06ac5a3904a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.012534 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87203be7-1191-409a-81d4-d06ac5a3904a admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.012659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87203be7-1191-409a-81d4-d06ac5a3904a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.013011 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87203be7-1191-409a-81d4-d06ac5a3904a admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.013598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87203be7-1191-409a-81d4-d06ac5a3904a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.015658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87203be7-1191-409a-81d4-d06ac5a3904a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.027037 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 783/1575] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:10:51.027949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0fcb5-b456-4660-a556-f40de75ad589 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.039132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0fcb5-b456-4660-a556-f40de75ad589 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.044100 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6ed8f-0567-4cca-9607-757b43605727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.044735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6ed8f-0567-4cca-9607-757b43605727 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.045022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6ed8f-0567-4cca-9607-757b43605727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.045334 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6ed8f-0567-4cca-9607-757b43605727 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.045998 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6ed8f-0567-4cca-9607-757b43605727 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.047928 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6ed8f-0567-4cca-9607-757b43605727 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.051721 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 793/1576] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:10:50 2022] POST /identity/v3/projects => generated 404 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:10:51.051721 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87203be7-1191-409a-81d4-d06ac5a3904a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.069814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 784/1577] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:10:51 2022] POST /identity/v3/users => generated 303 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:10:51.070642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6ed8f-0567-4cca-9607-757b43605727 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.084238 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6ed8f-0567-4cca-9607-757b43605727 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.124133 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87203be7-1191-409a-81d4-d06ac5a3904a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.128890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269bb1a-cbf9-4b89-aee7-ba2a5153a2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.129563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269bb1a-cbf9-4b89-aee7-ba2a5153a2ad admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.129853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269bb1a-cbf9-4b89-aee7-ba2a5153a2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.130396 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269bb1a-cbf9-4b89-aee7-ba2a5153a2ad admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.131070 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269bb1a-cbf9-4b89-aee7-ba2a5153a2ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.133011 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269bb1a-cbf9-4b89-aee7-ba2a5153a2ad admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.148288 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48e359-0487-4cb1-87b4-6885aa73f2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.148925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48e359-0487-4cb1-87b4-6885aa73f2fc admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.149243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48e359-0487-4cb1-87b4-6885aa73f2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.149525 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48e359-0487-4cb1-87b4-6885aa73f2fc admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.150198 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48e359-0487-4cb1-87b4-6885aa73f2fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.152134 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48e359-0487-4cb1-87b4-6885aa73f2fc admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.156270 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 794/1578] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:10:51 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:10:51.157156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269bb1a-cbf9-4b89-aee7-ba2a5153a2ad admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.169550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269bb1a-cbf9-4b89-aee7-ba2a5153a2ad admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.174227 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b8616-7722-4553-8ee9-856d668dbc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.174826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b8616-7722-4553-8ee9-856d668dbc5d admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.175106 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b8616-7722-4553-8ee9-856d668dbc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.175381 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b8616-7722-4553-8ee9-856d668dbc5d admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.176087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 785/1579] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:10:51 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:10:51.176216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b8616-7722-4553-8ee9-856d668dbc5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.177018 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48e359-0487-4cb1-87b4-6885aa73f2fc admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.178080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b8616-7722-4553-8ee9-856d668dbc5d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.187597 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48e359-0487-4cb1-87b4-6885aa73f2fc admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.193057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6fc61-280a-4474-9818-004905ed9091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.193716 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6fc61-280a-4474-9818-004905ed9091 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.193998 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6fc61-280a-4474-9818-004905ed9091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.194275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6fc61-280a-4474-9818-004905ed9091 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.195197 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6fc61-280a-4474-9818-004905ed9091 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.197252 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6fc61-280a-4474-9818-004905ed9091 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.201214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 795/1580] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:10:51 2022] POST /identity/v3/users => generated 296 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:10:51.201214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39334f53-7553-4264-922d-2214ada1991d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.201214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39334f53-7553-4264-922d-2214ada1991d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.201214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39334f53-7553-4264-922d-2214ada1991d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.220385 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 786/1581] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:10:51 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:10:51.221258 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6fc61-280a-4474-9818-004905ed9091 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.227723 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-39334f53-7553-4264-922d-2214ada1991d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:51.228927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-39334f53-7553-4264-922d-2214ada1991d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:51.231936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6fc61-280a-4474-9818-004905ed9091 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.236635 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25daca2f-1076-4cd6-9217-213c8e3fae2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.237096 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 796/1582] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:51.237917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39334f53-7553-4264-922d-2214ada1991d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.241275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25daca2f-1076-4cd6-9217-213c8e3fae2d admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.241275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25daca2f-1076-4cd6-9217-213c8e3fae2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.241275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25daca2f-1076-4cd6-9217-213c8e3fae2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.242091 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25daca2f-1076-4cd6-9217-213c8e3fae2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.242708 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:51.243813 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25daca2f-1076-4cd6-9217-213c8e3fae2d admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.245826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:51.248663 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39334f53-7553-4264-922d-2214ada1991d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.256852 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074cf4bd-b69e-43ed-9b1c-9c6ecf9b6ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.257547 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cf4bd-b69e-43ed-9b1c-9c6ecf9b6ea5 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.257832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cf4bd-b69e-43ed-9b1c-9c6ecf9b6ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.258302 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cf4bd-b69e-43ed-9b1c-9c6ecf9b6ea5 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.258459 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 787/1583] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:10:51 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:10:51.259188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074cf4bd-b69e-43ed-9b1c-9c6ecf9b6ea5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.259286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25daca2f-1076-4cd6-9217-213c8e3fae2d admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.261288 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074cf4bd-b69e-43ed-9b1c-9c6ecf9b6ea5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.272664 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25daca2f-1076-4cd6-9217-213c8e3fae2d admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.279353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0849f6-ccb5-4524-8612-8294b5b15d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.279905 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0849f6-ccb5-4524-8612-8294b5b15d41 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.280468 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0849f6-ccb5-4524-8612-8294b5b15d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.280723 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0849f6-ccb5-4524-8612-8294b5b15d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.281736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0849f6-ccb5-4524-8612-8294b5b15d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.283637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0849f6-ccb5-4524-8612-8294b5b15d41 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.290908 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 797/1584] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:10:51 2022] POST /identity/v3/users => generated 303 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:10:51.290908 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074cf4bd-b69e-43ed-9b1c-9c6ecf9b6ea5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.299440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074cf4bd-b69e-43ed-9b1c-9c6ecf9b6ea5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.301360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 788/1585] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:10:51 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:10:51.302318 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0849f6-ccb5-4524-8612-8294b5b15d41 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.305646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c5b0d-e28e-436c-a946-9fa9aef978ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.306301 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c5b0d-e28e-436c-a946-9fa9aef978ae admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.306587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c5b0d-e28e-436c-a946-9fa9aef978ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.306865 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c5b0d-e28e-436c-a946-9fa9aef978ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.307570 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c5b0d-e28e-436c-a946-9fa9aef978ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.309428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c5b0d-e28e-436c-a946-9fa9aef978ae admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.312419 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0849f6-ccb5-4524-8612-8294b5b15d41 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.317601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee551834-7674-4963-987b-c5858357b451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.319949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee551834-7674-4963-987b-c5858357b451 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.319949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee551834-7674-4963-987b-c5858357b451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.319949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee551834-7674-4963-987b-c5858357b451 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.319949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee551834-7674-4963-987b-c5858357b451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.321041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee551834-7674-4963-987b-c5858357b451 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.322113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 798/1586] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:10:51 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:10:51.322907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c5b0d-e28e-436c-a946-9fa9aef978ae admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.333358 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c5b0d-e28e-436c-a946-9fa9aef978ae admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.333573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 789/1587] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:10:51 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:10:51.335275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee551834-7674-4963-987b-c5858357b451 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.340013 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5125b6-0425-4121-8941-daa016176bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.340651 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5125b6-0425-4121-8941-daa016176bbc admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.340936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5125b6-0425-4121-8941-daa016176bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.341239 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5125b6-0425-4121-8941-daa016176bbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.341958 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5125b6-0425-4121-8941-daa016176bbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.343535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5125b6-0425-4121-8941-daa016176bbc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.344410 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee551834-7674-4963-987b-c5858357b451 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.349898 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb88d2-f5c5-4269-8f33-48b5b13cdcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.350501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb88d2-f5c5-4269-8f33-48b5b13cdcfb admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.350809 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb88d2-f5c5-4269-8f33-48b5b13cdcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.351088 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb88d2-f5c5-4269-8f33-48b5b13cdcfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.351776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb88d2-f5c5-4269-8f33-48b5b13cdcfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.353401 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb88d2-f5c5-4269-8f33-48b5b13cdcfb admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.354422 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 799/1588] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:10:51 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:10:51.355326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5125b6-0425-4121-8941-daa016176bbc admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.370158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 790/1589] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:10:51 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:10:51.370158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb88d2-f5c5-4269-8f33-48b5b13cdcfb admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.380605 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb88d2-f5c5-4269-8f33-48b5b13cdcfb admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.385839 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932cc62-a9ed-4651-9851-ccbd57eb4f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.386619 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932cc62-a9ed-4651-9851-ccbd57eb4f5d admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.387002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932cc62-a9ed-4651-9851-ccbd57eb4f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.387345 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932cc62-a9ed-4651-9851-ccbd57eb4f5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.388046 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932cc62-a9ed-4651-9851-ccbd57eb4f5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.389676 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932cc62-a9ed-4651-9851-ccbd57eb4f5d admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.402411 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 791/1590] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:10:51 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:10:51.403509 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932cc62-a9ed-4651-9851-ccbd57eb4f5d admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.413766 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932cc62-a9ed-4651-9851-ccbd57eb4f5d admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.418875 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb99f6a-9d34-4f54-a60b-7b03b1b43990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.419481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb99f6a-9d34-4f54-a60b-7b03b1b43990 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.419767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb99f6a-9d34-4f54-a60b-7b03b1b43990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.420044 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb99f6a-9d34-4f54-a60b-7b03b1b43990 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.420156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5125b6-0425-4121-8941-daa016176bbc admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.420767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb99f6a-9d34-4f54-a60b-7b03b1b43990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.422406 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb99f6a-9d34-4f54-a60b-7b03b1b43990 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.424233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8281b-4d36-409a-bd10-ddfdacb3602f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.424863 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8281b-4d36-409a-bd10-ddfdacb3602f service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.425197 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8281b-4d36-409a-bd10-ddfdacb3602f service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.425482 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8281b-4d36-409a-bd10-ddfdacb3602f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.432719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 792/1591] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:10:51 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:10:51.433595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb99f6a-9d34-4f54-a60b-7b03b1b43990 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.436506 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8281b-4d36-409a-bd10-ddfdacb3602f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.438013 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8281b-4d36-409a-bd10-ddfdacb3602f service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.443845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb99f6a-9d34-4f54-a60b-7b03b1b43990 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.448454 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6e12a-1d52-4e04-955c-16df5d76e4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.448623 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 800/1592] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:10:51 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:10:51.449170 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6e12a-1d52-4e04-955c-16df5d76e4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.449425 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8281b-4d36-409a-bd10-ddfdacb3602f service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.449543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6e12a-1d52-4e04-955c-16df5d76e4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.449728 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6e12a-1d52-4e04-955c-16df5d76e4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.450498 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6e12a-1d52-4e04-955c-16df5d76e4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.452098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6e12a-1d52-4e04-955c-16df5d76e4e7 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.459661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8281b-4d36-409a-bd10-ddfdacb3602f service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.464124 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 793/1593] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:10:51 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:10:51.465192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6e12a-1d52-4e04-955c-16df5d76e4e7 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.467862 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42156292-eee1-4490-a227-5c836b94c161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.468500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42156292-eee1-4490-a227-5c836b94c161 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.468785 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42156292-eee1-4490-a227-5c836b94c161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.469063 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42156292-eee1-4490-a227-5c836b94c161 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.470083 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42156292-eee1-4490-a227-5c836b94c161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.471429 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42156292-eee1-4490-a227-5c836b94c161 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.476080 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6e12a-1d52-4e04-955c-16df5d76e4e7 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.481021 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8df07e-09ea-4ae5-bf64-6476692ed012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.481672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8df07e-09ea-4ae5-bf64-6476692ed012 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.482034 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8df07e-09ea-4ae5-bf64-6476692ed012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.482262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8df07e-09ea-4ae5-bf64-6476692ed012 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.482994 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8df07e-09ea-4ae5-bf64-6476692ed012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.484215 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 801/1594] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:10:51 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:10:51.487536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8df07e-09ea-4ae5-bf64-6476692ed012 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.487825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42156292-eee1-4490-a227-5c836b94c161 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.496535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42156292-eee1-4490-a227-5c836b94c161 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.500799 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 794/1595] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:10:51 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:10:51.501844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8df07e-09ea-4ae5-bf64-6476692ed012 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.502356 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660b4e6-eca8-4090-a666-f9fbb04ea987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.503064 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660b4e6-eca8-4090-a666-f9fbb04ea987 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.503414 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660b4e6-eca8-4090-a666-f9fbb04ea987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.503761 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660b4e6-eca8-4090-a666-f9fbb04ea987 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.506009 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660b4e6-eca8-4090-a666-f9fbb04ea987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.507692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660b4e6-eca8-4090-a666-f9fbb04ea987 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.514413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8df07e-09ea-4ae5-bf64-6476692ed012 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.519362 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 802/1596] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:10:51 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:10:51.521429 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660b4e6-eca8-4090-a666-f9fbb04ea987 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.523531 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54132393-8433-4f4e-934c-9fe418308785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.524206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54132393-8433-4f4e-934c-9fe418308785 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.524491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54132393-8433-4f4e-934c-9fe418308785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.524767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54132393-8433-4f4e-934c-9fe418308785 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.525681 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54132393-8433-4f4e-934c-9fe418308785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.527303 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54132393-8433-4f4e-934c-9fe418308785 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.535011 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660b4e6-eca8-4090-a666-f9fbb04ea987 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.538246 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 795/1597] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:10:51 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:10:51.538915 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54132393-8433-4f4e-934c-9fe418308785 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.546846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b674a81-a808-4099-bd53-7cc7886019eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.547662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b674a81-a808-4099-bd53-7cc7886019eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.547948 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b674a81-a808-4099-bd53-7cc7886019eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.548324 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b674a81-a808-4099-bd53-7cc7886019eb admin admin] PATH_INFO: `/v3/projects/ada46409d8ae4abfaf19056675f7fbc6/users/0e93d159e8644203808918c5ae86fc74/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.551342 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54132393-8433-4f4e-934c-9fe418308785 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.556606 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5c749-53d1-4e0f-80cd-54f3f361c68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.557674 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5c749-53d1-4e0f-80cd-54f3f361c68a admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.557674 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5c749-53d1-4e0f-80cd-54f3f361c68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.557908 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5c749-53d1-4e0f-80cd-54f3f361c68a admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.558509 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5c749-53d1-4e0f-80cd-54f3f361c68a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.560291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5c749-53d1-4e0f-80cd-54f3f361c68a admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.570770 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b674a81-a808-4099-bd53-7cc7886019eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ada46409d8ae4abfaf19056675f7fbc6, user_id=0e93d159e8644203808918c5ae86fc74, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.573762 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b674a81-a808-4099-bd53-7cc7886019eb admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.575127 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 796/1598] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:10:51 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:10:51.579020 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5c749-53d1-4e0f-80cd-54f3f361c68a admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.590845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5c749-53d1-4e0f-80cd-54f3f361c68a admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.599597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 803/1599] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:10:51 2022] PUT /identity/v3/projects/ada46409d8ae4abfaf19056675f7fbc6/users/0e93d159e8644203808918c5ae86fc74/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:10:51.599597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b674a81-a808-4099-bd53-7cc7886019eb admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.689531 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74012cae-b489-49ff-be87-e9327403e3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.690221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74012cae-b489-49ff-be87-e9327403e3e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.690509 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74012cae-b489-49ff-be87-e9327403e3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.690789 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74012cae-b489-49ff-be87-e9327403e3e5 admin admin] PATH_INFO: `/v3/projects/baa2d28aea334dc9a029f30ec7799286/users/8f15ba47865645748f64f0135da67e4b/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.707814 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b674a81-a808-4099-bd53-7cc7886019eb admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.713596 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4d944-7e27-4d56-9470-454012cdcd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.714020 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74012cae-b489-49ff-be87-e9327403e3e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baa2d28aea334dc9a029f30ec7799286, user_id=8f15ba47865645748f64f0135da67e4b, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.714296 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4d944-7e27-4d56-9470-454012cdcd5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.714582 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4d944-7e27-4d56-9470-454012cdcd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.714872 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4d944-7e27-4d56-9470-454012cdcd5f admin admin] PATH_INFO: `/v3/projects/05905727ed6c41a1b6187fec6e61dd8f/users/c9633b9cbd8a4332a605ca66d73dc5b1/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.717714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74012cae-b489-49ff-be87-e9327403e3e5 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.733451 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4d944-7e27-4d56-9470-454012cdcd5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=05905727ed6c41a1b6187fec6e61dd8f, user_id=c9633b9cbd8a4332a605ca66d73dc5b1, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.736248 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4d944-7e27-4d56-9470-454012cdcd5f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.745794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 797/1600] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:10:51 2022] PUT /identity/v3/projects/baa2d28aea334dc9a029f30ec7799286/users/8f15ba47865645748f64f0135da67e4b/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:10:51.746654 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74012cae-b489-49ff-be87-e9327403e3e5 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.769203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 804/1601] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:10:51 2022] PUT /identity/v3/projects/05905727ed6c41a1b6187fec6e61dd8f/users/c9633b9cbd8a4332a605ca66d73dc5b1/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:10:51.770779 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4d944-7e27-4d56-9470-454012cdcd5f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.778318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:51.810793 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:51.830478 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74012cae-b489-49ff-be87-e9327403e3e5 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.871604 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4d944-7e27-4d56-9470-454012cdcd5f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:51.876926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edca63-e0ab-464e-b1d7-f2bc35fe5911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.877766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edca63-e0ab-464e-b1d7-f2bc35fe5911 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.878151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edca63-e0ab-464e-b1d7-f2bc35fe5911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.878531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edca63-e0ab-464e-b1d7-f2bc35fe5911 admin admin] PATH_INFO: `/v3/projects/279f631bf3db4ba7a0aff00226cf5724/users/6c30f2ab48e74d4690aafe46c4bf682f/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.898761 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90edca63-e0ab-464e-b1d7-f2bc35fe5911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=279f631bf3db4ba7a0aff00226cf5724, user_id=6c30f2ab48e74d4690aafe46c4bf682f, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.906175 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90edca63-e0ab-464e-b1d7-f2bc35fe5911 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.908016 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0f0af-fa36-4c79-98e7-5bb9ba221cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:51.908863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0f0af-fa36-4c79-98e7-5bb9ba221cb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:51.909429 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0f0af-fa36-4c79-98e7-5bb9ba221cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:51.909787 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0f0af-fa36-4c79-98e7-5bb9ba221cb0 admin admin] PATH_INFO: `/v3/projects/0d3b7509951d4beeacd7f741e7db83ba/users/9c5e4fd611a746639e1374e641721806/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:51.928066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 805/1602] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:10:51 2022] PUT /identity/v3/projects/279f631bf3db4ba7a0aff00226cf5724/users/6c30f2ab48e74d4690aafe46c4bf682f/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:10:51.928066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edca63-e0ab-464e-b1d7-f2bc35fe5911 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:51.932985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0f0af-fa36-4c79-98e7-5bb9ba221cb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d3b7509951d4beeacd7f741e7db83ba, user_id=9c5e4fd611a746639e1374e641721806, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:51.936197 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0f0af-fa36-4c79-98e7-5bb9ba221cb0 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:51.956144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 798/1603] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:10:51 2022] PUT /identity/v3/projects/0d3b7509951d4beeacd7f741e7db83ba/users/9c5e4fd611a746639e1374e641721806/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:10:51.956144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0f0af-fa36-4c79-98e7-5bb9ba221cb0 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:52.021765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edca63-e0ab-464e-b1d7-f2bc35fe5911 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:52.037974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0f0af-fa36-4c79-98e7-5bb9ba221cb0 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:52.045050 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a0ec0-92ab-4c01-a9d2-138983b6b330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:52.045813 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a0ec0-92ab-4c01-a9d2-138983b6b330 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:52.046107 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a0ec0-92ab-4c01-a9d2-138983b6b330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:52.046392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a0ec0-92ab-4c01-a9d2-138983b6b330 admin admin] PATH_INFO: `/v3/projects/3fb1da0ac63a44fcb786d3cd4be30142/users/cb07fc25243546ce87be2f456002147f/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:52.065645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a0ec0-92ab-4c01-a9d2-138983b6b330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fb1da0ac63a44fcb786d3cd4be30142, user_id=cb07fc25243546ce87be2f456002147f, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:52.068341 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a0ec0-92ab-4c01-a9d2-138983b6b330 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:52.085596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 799/1604] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:10:51 2022] PUT /identity/v3/projects/3fb1da0ac63a44fcb786d3cd4be30142/users/cb07fc25243546ce87be2f456002147f/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:10:52.087297 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6cf8c-4d00-446e-b3d3-b448af8e4688 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:52.087709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6cf8c-4d00-446e-b3d3-b448af8e4688 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:52.088001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6cf8c-4d00-446e-b3d3-b448af8e4688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:52.088308 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b5a57-d5d7-46a5-a09b-4042dc3144c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:52.089140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b5a57-d5d7-46a5-a09b-4042dc3144c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:52.089526 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b5a57-d5d7-46a5-a09b-4042dc3144c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:52.089903 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b5a57-d5d7-46a5-a09b-4042dc3144c3 admin admin] PATH_INFO: `/v3/projects/e44014cd67e24815b6783a7cebacc760/users/173aec63f5b845a9a282ffd4f106bcdb/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:52.109617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b5a57-d5d7-46a5-a09b-4042dc3144c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e44014cd67e24815b6783a7cebacc760, user_id=173aec63f5b845a9a282ffd4f106bcdb, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:52.110039 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-c0d6cf8c-4d00-446e-b3d3-b448af8e4688 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:52.111420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c0d6cf8c-4d00-446e-b3d3-b448af8e4688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:52.113300 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b5a57-d5d7-46a5-a09b-4042dc3144c3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:52.130844 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 806/1605] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:10:51 2022] PUT /identity/v3/projects/e44014cd67e24815b6783a7cebacc760/users/173aec63f5b845a9a282ffd4f106bcdb/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:10:52.132141 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064244b2-839d-4209-aaff-d93efc8c5ddf None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:52.132430 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064244b2-839d-4209-aaff-d93efc8c5ddf None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:52.132707 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064244b2-839d-4209-aaff-d93efc8c5ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:52.170871 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-064244b2-839d-4209-aaff-d93efc8c5ddf None None] MFA Rules not processed for user `8f15ba47865645748f64f0135da67e4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:52.172491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-064244b2-839d-4209-aaff-d93efc8c5ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:52.173793 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:52.179503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:52.201505 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 800/1606] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:52.203339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c68cf6-cb5f-43e6-957a-e8b18f9f5f46 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:52.203805 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c68cf6-cb5f-43e6-957a-e8b18f9f5f46 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:52.204542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c68cf6-cb5f-43e6-957a-e8b18f9f5f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:52.229354 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-a1c68cf6-cb5f-43e6-957a-e8b18f9f5f46 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:52.230749 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-a1c68cf6-cb5f-43e6-957a-e8b18f9f5f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:52.273601 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 807/1607] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:52 2022] POST /identity/v3/auth/tokens => generated 3290 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:52.275253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f6982-c951-4302-8104-a0242926c9cd None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:52.275742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f6982-c951-4302-8104-a0242926c9cd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:52.276168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f6982-c951-4302-8104-a0242926c9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:52.297809 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-7e0f6982-c951-4302-8104-a0242926c9cd None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:52.299056 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7e0f6982-c951-4302-8104-a0242926c9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:52.358313 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 801/1608] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:52.359944 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14591671-1ddf-464c-9620-07b0253c1851 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:52.360523 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14591671-1ddf-464c-9620-07b0253c1851 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:52.361037 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14591671-1ddf-464c-9620-07b0253c1851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:52.390212 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 808/1609] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:52.393141 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41821a9-2442-42f8-aea5-c448043833c4 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:52.393444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41821a9-2442-42f8-aea5-c448043833c4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:52.393885 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41821a9-2442-42f8-aea5-c448043833c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:52.419107 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-14591671-1ddf-464c-9620-07b0253c1851 None None] MFA Rules not processed for user `9c5e4fd611a746639e1374e641721806`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:52.419489 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a41821a9-2442-42f8-aea5-c448043833c4 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:52.420720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a41821a9-2442-42f8-aea5-c448043833c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:52.421597 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-14591671-1ddf-464c-9620-07b0253c1851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:52.433289 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 809/1610] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:52.434897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c1d6e-83f6-4718-91d6-95856089440c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:52.435193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c1d6e-83f6-4718-91d6-95856089440c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:52.435475 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c1d6e-83f6-4718-91d6-95856089440c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:52.483763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-063c1d6e-83f6-4718-91d6-95856089440c None None] MFA Rules not processed for user `173aec63f5b845a9a282ffd4f106bcdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:52.486420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-063c1d6e-83f6-4718-91d6-95856089440c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:52.556593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 802/1611] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:52.557851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14591671-1ddf-464c-9620-07b0253c1851 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:52.623871 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 810/1612] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:52.625208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c1d6e-83f6-4718-91d6-95856089440c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:52.639888 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14591671-1ddf-464c-9620-07b0253c1851 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:52.641466 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c1d6e-83f6-4718-91d6-95856089440c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:52.646056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506add2-997e-4523-80cd-979fb46fe08a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:52.646805 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506add2-997e-4523-80cd-979fb46fe08a service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:52.647091 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506add2-997e-4523-80cd-979fb46fe08a service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:52.647370 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506add2-997e-4523-80cd-979fb46fe08a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:52.647605 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32693426-2462-41f1-8a41-a507b7356ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:52.648251 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32693426-2462-41f1-8a41-a507b7356ecf admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:52.648537 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32693426-2462-41f1-8a41-a507b7356ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:52.648816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32693426-2462-41f1-8a41-a507b7356ecf admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:52.649541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32693426-2462-41f1-8a41-a507b7356ecf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:52.651657 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32693426-2462-41f1-8a41-a507b7356ecf admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:52.663087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506add2-997e-4523-80cd-979fb46fe08a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:52.664708 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506add2-997e-4523-80cd-979fb46fe08a service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:52.677222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 803/1613] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:10:52 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:10:52.678967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506add2-997e-4523-80cd-979fb46fe08a service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:52.690338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506add2-997e-4523-80cd-979fb46fe08a service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:52.695892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ecc9f-057d-4a6d-bf8b-6e8125d90b1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:52.696629 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ecc9f-057d-4a6d-bf8b-6e8125d90b1c service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:52.696969 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ecc9f-057d-4a6d-bf8b-6e8125d90b1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:52.697401 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ecc9f-057d-4a6d-bf8b-6e8125d90b1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:52.712124 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ecc9f-057d-4a6d-bf8b-6e8125d90b1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:52.714149 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ecc9f-057d-4a6d-bf8b-6e8125d90b1c service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:52.729442 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 811/1614] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:10:52 2022] POST /identity/v3/projects => generated 406 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:10:52.733346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32693426-2462-41f1-8a41-a507b7356ecf admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:52.740444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 804/1615] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:10:52 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:10:52.741838 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ecc9f-057d-4a6d-bf8b-6e8125d90b1c service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:52.753072 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ecc9f-057d-4a6d-bf8b-6e8125d90b1c service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:52.763901 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:52.767789 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:52.833978 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408b7f3-3ba0-4c1a-a1dd-85ec1e0ae83f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:52.834651 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408b7f3-3ba0-4c1a-a1dd-85ec1e0ae83f service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:52.834942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408b7f3-3ba0-4c1a-a1dd-85ec1e0ae83f service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:52.835226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408b7f3-3ba0-4c1a-a1dd-85ec1e0ae83f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:52.846744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32693426-2462-41f1-8a41-a507b7356ecf admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:52.851917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a2a38-d414-446e-97c0-c023f51ecbe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:52.852412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a2a38-d414-446e-97c0-c023f51ecbe0 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:52.852705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a2a38-d414-446e-97c0-c023f51ecbe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:52.852985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a2a38-d414-446e-97c0-c023f51ecbe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:52.937031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408b7f3-3ba0-4c1a-a1dd-85ec1e0ae83f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:52.938570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408b7f3-3ba0-4c1a-a1dd-85ec1e0ae83f service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:52.954787 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 805/1616] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:10:52 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:10:52.956411 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059862e-8635-42c9-a933-48b90b512f95 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:52.956748 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059862e-8635-42c9-a933-48b90b512f95 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:52.957692 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059862e-8635-42c9-a933-48b90b512f95 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:52.960534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 806/1617] 10.222.0.100 () {54 vars in 967 bytes} [Tue Aug 2 09:10:52 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:10:52.962197 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd9e31-d9e4-46a5-a2c2-4a5ef5a0a9b2 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:52.962777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd9e31-d9e4-46a5-a2c2-4a5ef5a0a9b2 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:52.963069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd9e31-d9e4-46a5-a2c2-4a5ef5a0a9b2 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:52.964964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 807/1618] 10.222.0.100 () {54 vars in 967 bytes} [Tue Aug 2 09:10:52 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:10:52.965788 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd9e31-d9e4-46a5-a2c2-4a5ef5a0a9b2 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:52.986013 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd9e31-d9e4-46a5-a2c2-4a5ef5a0a9b2 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:52.992439 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420dfcc5-87f1-4d92-acf8-1cf4cc13152a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:52.993366 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dfcc5-87f1-4d92-acf8-1cf4cc13152a admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:52.993657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dfcc5-87f1-4d92-acf8-1cf4cc13152a admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:52.993945 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dfcc5-87f1-4d92-acf8-1cf4cc13152a admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:52.996416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420dfcc5-87f1-4d92-acf8-1cf4cc13152a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:52.997614 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420dfcc5-87f1-4d92-acf8-1cf4cc13152a admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:53.017701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a2a38-d414-446e-97c0-c023f51ecbe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:53.019237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a2a38-d414-446e-97c0-c023f51ecbe0 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:53.039725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 808/1619] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:10:52 2022] POST /identity/v3/users => generated 310 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:10:53.041420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 812/1620] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:10:52 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:10:53.044712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a79d7-b1b4-4d39-addc-bcadc5f3a992 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:53.045137 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a79d7-b1b4-4d39-addc-bcadc5f3a992 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:53.045630 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a79d7-b1b4-4d39-addc-bcadc5f3a992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:53.045777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8850178-2401-400e-8f2b-116486531098 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:53.046014 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8850178-2401-400e-8f2b-116486531098 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:53.046305 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8850178-2401-400e-8f2b-116486531098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:53.111018 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-e8850178-2401-400e-8f2b-116486531098 None None] MFA Rules not processed for user `49a0c1a4a6cf43e28ec1b8488c681848`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:53.112631 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e8850178-2401-400e-8f2b-116486531098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:53.125647 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-350a79d7-b1b4-4d39-addc-bcadc5f3a992 None None] MFA Rules not processed for user `49a0c1a4a6cf43e28ec1b8488c681848`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:53.132222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-350a79d7-b1b4-4d39-addc-bcadc5f3a992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:53.235327 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 809/1621] 10.222.0.100 () {58 vars in 1064 bytes} [Tue Aug 2 09:10:53 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:53.236388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8850178-2401-400e-8f2b-116486531098 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:53.252908 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8850178-2401-400e-8f2b-116486531098 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:53.263506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef9c34-d1f2-476c-9bc2-2361c3543d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:53.265313 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef9c34-d1f2-476c-9bc2-2361c3543d16 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:53.265615 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef9c34-d1f2-476c-9bc2-2361c3543d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:53.265926 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef9c34-d1f2-476c-9bc2-2361c3543d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:53.267239 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef9c34-d1f2-476c-9bc2-2361c3543d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:53.271375 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef9c34-d1f2-476c-9bc2-2361c3543d16 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:53.294224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 810/1622] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:10:53 2022] GET /identity/v3/roles => generated 1427 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:10:53.296035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef9c34-d1f2-476c-9bc2-2361c3543d16 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:53.317305 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef9c34-d1f2-476c-9bc2-2361c3543d16 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:53.326485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062079d-76c3-46da-8c5a-364001354ed6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:53.327224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062079d-76c3-46da-8c5a-364001354ed6 service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:53.328330 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062079d-76c3-46da-8c5a-364001354ed6 service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:53.328643 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062079d-76c3-46da-8c5a-364001354ed6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:53.342837 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:53.391901 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 813/1623] 10.222.0.100 () {58 vars in 1064 bytes} [Tue Aug 2 09:10:53 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:53.409314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a79d7-b1b4-4d39-addc-bcadc5f3a992 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:53.424504 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062079d-76c3-46da-8c5a-364001354ed6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:53.428029 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062079d-76c3-46da-8c5a-364001354ed6 service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:53.444289 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 811/1624] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:10:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:10:53.445781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062079d-76c3-46da-8c5a-364001354ed6 service nova] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:53.447339 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a79d7-b1b4-4d39-addc-bcadc5f3a992 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:53.459939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf270a5-9f82-4cb8-a058-3e6a84c54524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:53.460711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf270a5-9f82-4cb8-a058-3e6a84c54524 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:53.463255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062079d-76c3-46da-8c5a-364001354ed6 service nova] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:53.464529 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf270a5-9f82-4cb8-a058-3e6a84c54524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:53.465098 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf270a5-9f82-4cb8-a058-3e6a84c54524 admin admin] PATH_INFO: `/v3/projects/78e20dede1f648e4ac0dd0ecca6665b9/users/529ee7f2c9844888a5bb68108aec4c1a/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:53.491603 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f10e2-17c8-4fe7-a2b8-fc77bbad1cf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:53.492685 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f10e2-17c8-4fe7-a2b8-fc77bbad1cf0 service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:53.494876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f10e2-17c8-4fe7-a2b8-fc77bbad1cf0 service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:53.495707 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f10e2-17c8-4fe7-a2b8-fc77bbad1cf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:53.512242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf270a5-9f82-4cb8-a058-3e6a84c54524 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78e20dede1f648e4ac0dd0ecca6665b9, user_id=529ee7f2c9844888a5bb68108aec4c1a, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:53.515068 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf270a5-9f82-4cb8-a058-3e6a84c54524 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:53.545042 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 814/1625] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:10:53 2022] PUT /identity/v3/projects/78e20dede1f648e4ac0dd0ecca6665b9/users/529ee7f2c9844888a5bb68108aec4c1a/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:10:53.549559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf270a5-9f82-4cb8-a058-3e6a84c54524 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:53.574133 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:53.647272 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f10e2-17c8-4fe7-a2b8-fc77bbad1cf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:53.653288 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf270a5-9f82-4cb8-a058-3e6a84c54524 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:53.654309 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f10e2-17c8-4fe7-a2b8-fc77bbad1cf0 service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:53.687423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f950d5d-63e3-4e2c-8dbb-f8c7dca42a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:53.688672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f950d5d-63e3-4e2c-8dbb-f8c7dca42a66 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:53.688971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f950d5d-63e3-4e2c-8dbb-f8c7dca42a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:53.689285 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f950d5d-63e3-4e2c-8dbb-f8c7dca42a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:53.690021 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f950d5d-63e3-4e2c-8dbb-f8c7dca42a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:53.701178 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f950d5d-63e3-4e2c-8dbb-f8c7dca42a66 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:53.712226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 812/1626] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:10:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:10:53.722176 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 815/1627] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:10:53 2022] GET /identity/v3/roles => generated 1427 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:10:53.728011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f10e2-17c8-4fe7-a2b8-fc77bbad1cf0 service nova] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:53.763108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f10e2-17c8-4fe7-a2b8-fc77bbad1cf0 service nova] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:53.786370 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2e63e-1396-4940-b0b7-5894c3fc8e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:53.788752 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2e63e-1396-4940-b0b7-5894c3fc8e75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:53.789204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2e63e-1396-4940-b0b7-5894c3fc8e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:53.789593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2e63e-1396-4940-b0b7-5894c3fc8e75 admin admin] PATH_INFO: `/v3/projects/78e20dede1f648e4ac0dd0ecca6665b9/users/529ee7f2c9844888a5bb68108aec4c1a/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:53.797371 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2e63e-1396-4940-b0b7-5894c3fc8e75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78e20dede1f648e4ac0dd0ecca6665b9, user_id=529ee7f2c9844888a5bb68108aec4c1a, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:53.804429 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2e63e-1396-4940-b0b7-5894c3fc8e75 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:53.906325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 813/1628] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:10:53 2022] PUT /identity/v3/projects/78e20dede1f648e4ac0dd0ecca6665b9/users/529ee7f2c9844888a5bb68108aec4c1a/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:10:53.911798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cb705-d52d-4337-b837-61918b9d6bc1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:53.911798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cb705-d52d-4337-b837-61918b9d6bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:53.911798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cb705-d52d-4337-b837-61918b9d6bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:53.912621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41991757-695f-4e7c-8277-17b1f6d3f87f None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:53.912621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41991757-695f-4e7c-8277-17b1f6d3f87f None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:53.912621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41991757-695f-4e7c-8277-17b1f6d3f87f None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:53.912621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 814/1629] 10.222.0.100 () {54 vars in 917 bytes} [Tue Aug 2 09:10:53 2022] GET /identity => generated 271 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:10:53.922161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b93ca0-fff1-4bd7-92ab-f91267f53627 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:53.922161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b93ca0-fff1-4bd7-92ab-f91267f53627 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:53.922657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b93ca0-fff1-4bd7-92ab-f91267f53627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:53.971736 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-737cb705-d52d-4337-b837-61918b9d6bc1 None None] MFA Rules not processed for user `529ee7f2c9844888a5bb68108aec4c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:53.977283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-737cb705-d52d-4337-b837-61918b9d6bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:54.025916 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-b2b93ca0-fff1-4bd7-92ab-f91267f53627 None None] MFA Rules not processed for user `49a0c1a4a6cf43e28ec1b8488c681848`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:54.029183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b2b93ca0-fff1-4bd7-92ab-f91267f53627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:54.181823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 816/1630] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:53 2022] POST /identity/v3/auth/tokens => generated 3359 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:54.186252 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cb705-d52d-4337-b837-61918b9d6bc1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:54.225395 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 815/1631] 10.222.0.100 () {58 vars in 1014 bytes} [Tue Aug 2 09:10:53 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:10:54.252394 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b93ca0-fff1-4bd7-92ab-f91267f53627 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:54.325644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:54.355611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cb705-d52d-4337-b837-61918b9d6bc1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:54.365612 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076fd0a-80cb-4a12-8a5b-8adf9bfcd01a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:54.366852 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076fd0a-80cb-4a12-8a5b-8adf9bfcd01a service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:54.367352 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076fd0a-80cb-4a12-8a5b-8adf9bfcd01a service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:54.368233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076fd0a-80cb-4a12-8a5b-8adf9bfcd01a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:54.383102 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076fd0a-80cb-4a12-8a5b-8adf9bfcd01a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:54.385930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076fd0a-80cb-4a12-8a5b-8adf9bfcd01a service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:54.395296 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b93ca0-fff1-4bd7-92ab-f91267f53627 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:54.401717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1209a-8a02-4576-ba4b-8bac66718fc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:54.402371 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1209a-8a02-4576-ba4b-8bac66718fc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:54.402722 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1209a-8a02-4576-ba4b-8bac66718fc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:54.402996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1209a-8a02-4576-ba4b-8bac66718fc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:54.405675 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 817/1632] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:10:54 2022] GET /identity/v3/auth/tokens => generated 3359 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:10:54.410821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d9eaa-1ee8-46ef-84ed-a2abc4c45c8e None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:54.412946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d9eaa-1ee8-46ef-84ed-a2abc4c45c8e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:54.414146 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d9eaa-1ee8-46ef-84ed-a2abc4c45c8e None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:54.417894 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 818/1633] 10.222.0.100 () {54 vars in 917 bytes} [Tue Aug 2 09:10:54 2022] GET /identity => generated 271 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:10:54.423485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1209a-8a02-4576-ba4b-8bac66718fc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:54.429824 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1209a-8a02-4576-ba4b-8bac66718fc5 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:54.434073 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67212b-8c06-444e-a83e-6250c973eb34 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:54.434579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67212b-8c06-444e-a83e-6250c973eb34 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:54.435013 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67212b-8c06-444e-a83e-6250c973eb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:54.462784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 816/1634] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:10:54 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:10:54.476357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-0b67212b-8c06-444e-a83e-6250c973eb34 None None] MFA Rules not processed for user `49a0c1a4a6cf43e28ec1b8488c681848`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:54.477771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0b67212b-8c06-444e-a83e-6250c973eb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:54.504856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 819/1635] 10.222.0.100 () {58 vars in 1014 bytes} [Tue Aug 2 09:10:54 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:54.562219 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1209a-8a02-4576-ba4b-8bac66718fc5 service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:54.600025 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1209a-8a02-4576-ba4b-8bac66718fc5 service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:54.622315 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010dc203-5297-4e4e-a775-e974da9870c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:54.623171 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dc203-5297-4e4e-a775-e974da9870c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:54.623703 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dc203-5297-4e4e-a775-e974da9870c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:54.624058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dc203-5297-4e4e-a775-e974da9870c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:54.656491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010dc203-5297-4e4e-a775-e974da9870c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:54.658189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010dc203-5297-4e4e-a775-e974da9870c6 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:54.698739 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 817/1636] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:10:54 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:10:55.138437 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67212b-8c06-444e-a83e-6250c973eb34 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:55.229744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:55.248230 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010dc203-5297-4e4e-a775-e974da9870c6 service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:55.283070 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67212b-8c06-444e-a83e-6250c973eb34 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:55.290539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee738409-f135-4f90-adaa-6e2ee14e43f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:55.291646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee738409-f135-4f90-adaa-6e2ee14e43f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:55.291646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee738409-f135-4f90-adaa-6e2ee14e43f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:55.292043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee738409-f135-4f90-adaa-6e2ee14e43f5 admin admin] PATH_INFO: `/v3/users/8f15ba47865645748f64f0135da67e4b` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:55.293972 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee738409-f135-4f90-adaa-6e2ee14e43f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f15ba47865645748f64f0135da67e4b)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:55.296084 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee738409-f135-4f90-adaa-6e2ee14e43f5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:55.413925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010dc203-5297-4e4e-a775-e974da9870c6 service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:55.423862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae603c93-ea3d-4453-beb1-dea4fb7c60ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:55.424645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae603c93-ea3d-4453-beb1-dea4fb7c60ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:55.424940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae603c93-ea3d-4453-beb1-dea4fb7c60ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:55.425371 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae603c93-ea3d-4453-beb1-dea4fb7c60ed admin admin] PATH_INFO: `/v3/users/173aec63f5b845a9a282ffd4f106bcdb` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:55.428588 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae603c93-ea3d-4453-beb1-dea4fb7c60ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=173aec63f5b845a9a282ffd4f106bcdb)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:55.429716 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: WARNING py.warnings [None req-ae603c93-ea3d-4453-beb1-dea4fb7c60ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:10:55.429716 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: warnings.warn(msg) Aug 02 09:10:55.429716 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: Aug 02 09:10:55.436291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae603c93-ea3d-4453-beb1-dea4fb7c60ed admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:55.438222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ee738409-f135-4f90-adaa-6e2ee14e43f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f15ba47865645748f64f0135da67e4b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:55.464319 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ee738409-f135-4f90-adaa-6e2ee14e43f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f15ba47865645748f64f0135da67e4b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:55.467443 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ee738409-f135-4f90-adaa-6e2ee14e43f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f15ba47865645748f64f0135da67e4b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:55.490982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ee738409-f135-4f90-adaa-6e2ee14e43f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f15ba47865645748f64f0135da67e4b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:55.492367 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ee738409-f135-4f90-adaa-6e2ee14e43f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f15ba47865645748f64f0135da67e4b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:55.578392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ae603c93-ea3d-4453-beb1-dea4fb7c60ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '173aec63f5b845a9a282ffd4f106bcdb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:55.579092 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ae603c93-ea3d-4453-beb1-dea4fb7c60ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '173aec63f5b845a9a282ffd4f106bcdb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:55.596271 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ae603c93-ea3d-4453-beb1-dea4fb7c60ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '173aec63f5b845a9a282ffd4f106bcdb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:55.596814 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 820/1637] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:10:55 2022] DELETE /identity/v3/users/8f15ba47865645748f64f0135da67e4b => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:10:55.602758 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfb663-75f3-4d26-b88d-aab0ea61ce19 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:55.603960 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfb663-75f3-4d26-b88d-aab0ea61ce19 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:55.604293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfb663-75f3-4d26-b88d-aab0ea61ce19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:55.647262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ae603c93-ea3d-4453-beb1-dea4fb7c60ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '173aec63f5b845a9a282ffd4f106bcdb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:55.661171 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-4cdfb663-75f3-4d26-b88d-aab0ea61ce19 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:55.666176 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4cdfb663-75f3-4d26-b88d-aab0ea61ce19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:55.681135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ae603c93-ea3d-4453-beb1-dea4fb7c60ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '173aec63f5b845a9a282ffd4f106bcdb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:55.695094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 818/1638] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:10:55 2022] DELETE /identity/v3/users/173aec63f5b845a9a282ffd4f106bcdb => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:10:55.702300 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a2876-9227-4c3e-905d-33a6f9fb709e None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:55.702586 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a2876-9227-4c3e-905d-33a6f9fb709e None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:55.702859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a2876-9227-4c3e-905d-33a6f9fb709e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:55.750790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-a32a2876-9227-4c3e-905d-33a6f9fb709e None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:55.752044 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-a32a2876-9227-4c3e-905d-33a6f9fb709e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:55.850364 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:55.929966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 819/1639] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:55.930667 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe80b5-1cfc-40f7-97ca-81a9035b0bcf None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:55.930667 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe80b5-1cfc-40f7-97ca-81a9035b0bcf None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:55.930892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe80b5-1cfc-40f7-97ca-81a9035b0bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:56.002316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-30fe80b5-1cfc-40f7-97ca-81a9035b0bcf None None] MFA Rules not processed for user `cb07fc25243546ce87be2f456002147f`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:56.003967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-30fe80b5-1cfc-40f7-97ca-81a9035b0bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:56.037626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 821/1640] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:56.040215 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfb663-75f3-4d26-b88d-aab0ea61ce19 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:56.042532 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4cdfb663-75f3-4d26-b88d-aab0ea61ce19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:56.153535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:56.162656 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 820/1641] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:55 2022] POST /identity/v3/auth/tokens => generated 3290 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:56.162656 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe80b5-1cfc-40f7-97ca-81a9035b0bcf None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:56.172100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfb663-75f3-4d26-b88d-aab0ea61ce19 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:56.325505 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe80b5-1cfc-40f7-97ca-81a9035b0bcf None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:56.327549 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449762e-7418-4a34-a739-911b286929d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:56.330358 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449762e-7418-4a34-a739-911b286929d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:56.330898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449762e-7418-4a34-a739-911b286929d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:56.331776 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449762e-7418-4a34-a739-911b286929d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:56.339052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755ae36-f015-4f2b-a11e-cafd13bf88cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:56.340725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755ae36-f015-4f2b-a11e-cafd13bf88cc service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:56.340725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755ae36-f015-4f2b-a11e-cafd13bf88cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:56.340725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755ae36-f015-4f2b-a11e-cafd13bf88cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:56.356157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755ae36-f015-4f2b-a11e-cafd13bf88cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:56.358748 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755ae36-f015-4f2b-a11e-cafd13bf88cc service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:56.358919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449762e-7418-4a34-a739-911b286929d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:56.360456 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449762e-7418-4a34-a739-911b286929d0 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:56.402277 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 821/1642] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:10:56 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:10:56.404032 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755ae36-f015-4f2b-a11e-cafd13bf88cc service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:56.404929 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 822/1643] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:10:56 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:10:56.412445 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-4755ae36-f015-4f2b-a11e-cafd13bf88cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:56.433465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755ae36-f015-4f2b-a11e-cafd13bf88cc service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:56.440431 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeab9e3-7086-48a7-9a77-69e6ebe6343f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:56.441990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeab9e3-7086-48a7-9a77-69e6ebe6343f admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:56.442532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeab9e3-7086-48a7-9a77-69e6ebe6343f admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:56.443070 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeab9e3-7086-48a7-9a77-69e6ebe6343f admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:56.446393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeab9e3-7086-48a7-9a77-69e6ebe6343f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:56.450942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeab9e3-7086-48a7-9a77-69e6ebe6343f admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:56.524283 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 822/1644] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:10:56 2022] POST /identity/v3/projects => generated 404 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:10:56.528750 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449762e-7418-4a34-a739-911b286929d0 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:56.542743 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449762e-7418-4a34-a739-911b286929d0 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:56.665891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6f71d-7a90-4e2a-8d2c-93af4d00e626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:56.666611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6f71d-7a90-4e2a-8d2c-93af4d00e626 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:56.666899 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6f71d-7a90-4e2a-8d2c-93af4d00e626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:56.667253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6f71d-7a90-4e2a-8d2c-93af4d00e626 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:56.667890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6f71d-7a90-4e2a-8d2c-93af4d00e626 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:56.674385 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6f71d-7a90-4e2a-8d2c-93af4d00e626 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:56.709367 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 823/1645] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:10:56 2022] POST /identity/v3/users => generated 309 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:10:56.713286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeab9e3-7086-48a7-9a77-69e6ebe6343f admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:56.728716 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeab9e3-7086-48a7-9a77-69e6ebe6343f admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:56.742862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b695dd-cc75-4dde-a229-e982e44c1ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:56.743581 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b695dd-cc75-4dde-a229-e982e44c1ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:56.743930 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b695dd-cc75-4dde-a229-e982e44c1ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:56.744398 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b695dd-cc75-4dde-a229-e982e44c1ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:56.745273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b695dd-cc75-4dde-a229-e982e44c1ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:56.747432 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b695dd-cc75-4dde-a229-e982e44c1ed4 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:56.784096 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 823/1646] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:10:56 2022] GET /identity/v3/roles => generated 1427 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:10:56.785451 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6f71d-7a90-4e2a-8d2c-93af4d00e626 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:56.821585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6f71d-7a90-4e2a-8d2c-93af4d00e626 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:56.830200 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930c29c-738b-47d1-9a90-c6dd54dd1a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:56.831678 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930c29c-738b-47d1-9a90-c6dd54dd1a16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:56.833253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930c29c-738b-47d1-9a90-c6dd54dd1a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:56.833605 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930c29c-738b-47d1-9a90-c6dd54dd1a16 admin admin] PATH_INFO: `/v3/projects/eae1791d3b4948afaf496d147ef2906b/users/5bcbbb365c124b1d9987c31315571409/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:56.861371 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930c29c-738b-47d1-9a90-c6dd54dd1a16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae1791d3b4948afaf496d147ef2906b, user_id=5bcbbb365c124b1d9987c31315571409, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:56.868127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930c29c-738b-47d1-9a90-c6dd54dd1a16 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:56.898914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 824/1647] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:10:56 2022] PUT /identity/v3/projects/eae1791d3b4948afaf496d147ef2906b/users/5bcbbb365c124b1d9987c31315571409/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:10:56.907080 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b695dd-cc75-4dde-a229-e982e44c1ed4 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:56.925554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b695dd-cc75-4dde-a229-e982e44c1ed4 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:56.978568 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:57.062381 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec69304-df13-450b-bb32-c15d349bbc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:57.063363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec69304-df13-450b-bb32-c15d349bbc99 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:57.063906 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec69304-df13-450b-bb32-c15d349bbc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:57.064495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec69304-df13-450b-bb32-c15d349bbc99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:57.065513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec69304-df13-450b-bb32-c15d349bbc99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:57.069845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec69304-df13-450b-bb32-c15d349bbc99 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:57.089569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930c29c-738b-47d1-9a90-c6dd54dd1a16 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:57.090880 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 824/1648] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:10:56 2022] GET /identity/v3/roles => generated 1427 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:10:57.102141 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930c29c-738b-47d1-9a90-c6dd54dd1a16 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:57.134753 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918824f-a033-4698-b78d-91dcb7c6954a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:57.143948 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918824f-a033-4698-b78d-91dcb7c6954a admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:57.146152 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918824f-a033-4698-b78d-91dcb7c6954a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:57.146315 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918824f-a033-4698-b78d-91dcb7c6954a admin admin] PATH_INFO: `/v3/projects/eae1791d3b4948afaf496d147ef2906b/users/5bcbbb365c124b1d9987c31315571409/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:57.166153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918824f-a033-4698-b78d-91dcb7c6954a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae1791d3b4948afaf496d147ef2906b, user_id=5bcbbb365c124b1d9987c31315571409, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:57.168857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918824f-a033-4698-b78d-91dcb7c6954a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:57.186589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 825/1649] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:10:57 2022] PUT /identity/v3/projects/eae1791d3b4948afaf496d147ef2906b/users/5bcbbb365c124b1d9987c31315571409/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:10:58.589866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec69304-df13-450b-bb32-c15d349bbc99 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:58.592125 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-dec69304-df13-450b-bb32-c15d349bbc99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:58.616455 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec69304-df13-450b-bb32-c15d349bbc99 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:58.796496 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735ac11-b57a-4c15-89bd-44fc31bbb063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:58.797386 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735ac11-b57a-4c15-89bd-44fc31bbb063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:58.797858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735ac11-b57a-4c15-89bd-44fc31bbb063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:58.798386 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735ac11-b57a-4c15-89bd-44fc31bbb063 admin admin] PATH_INFO: `/v3/users/9c5e4fd611a746639e1374e641721806` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:58.800603 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735ac11-b57a-4c15-89bd-44fc31bbb063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c5e4fd611a746639e1374e641721806)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:58.809430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735ac11-b57a-4c15-89bd-44fc31bbb063 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:58.906016 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-4735ac11-b57a-4c15-89bd-44fc31bbb063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c5e4fd611a746639e1374e641721806'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:58.906750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-4735ac11-b57a-4c15-89bd-44fc31bbb063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c5e4fd611a746639e1374e641721806'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:58.926839 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-4735ac11-b57a-4c15-89bd-44fc31bbb063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c5e4fd611a746639e1374e641721806'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:58.950802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-4735ac11-b57a-4c15-89bd-44fc31bbb063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c5e4fd611a746639e1374e641721806'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:58.971484 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-4735ac11-b57a-4c15-89bd-44fc31bbb063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c5e4fd611a746639e1374e641721806'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:58.986588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766c38a-8818-4f4d-a334-2135d22bad8e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:58.987436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766c38a-8818-4f4d-a334-2135d22bad8e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:58.987436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766c38a-8818-4f4d-a334-2135d22bad8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:58.989888 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 825/1650] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:10:58 2022] DELETE /identity/v3/users/9c5e4fd611a746639e1374e641721806 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:10:59.009760 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6766c38a-8818-4f4d-a334-2135d22bad8e None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:59.011034 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6766c38a-8818-4f4d-a334-2135d22bad8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:59.118770 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:59.133592 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 826/1651] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:10:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:59.174463 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735ac11-b57a-4c15-89bd-44fc31bbb063 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:59.177391 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-4735ac11-b57a-4c15-89bd-44fc31bbb063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:59.322362 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735ac11-b57a-4c15-89bd-44fc31bbb063 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:59.331319 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6820e3-4d3f-48a6-9298-9e33a728fdb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:59.331989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6820e3-4d3f-48a6-9298-9e33a728fdb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:59.332400 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6820e3-4d3f-48a6-9298-9e33a728fdb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:59.332788 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6820e3-4d3f-48a6-9298-9e33a728fdb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:59.356698 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:59.362576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6820e3-4d3f-48a6-9298-9e33a728fdb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:59.364725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6820e3-4d3f-48a6-9298-9e33a728fdb0 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:59.387257 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 826/1652] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:10:59 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:10:59.976151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766c38a-8818-4f4d-a334-2135d22bad8e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:59.977889 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6766c38a-8818-4f4d-a334-2135d22bad8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:59.997993 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766c38a-8818-4f4d-a334-2135d22bad8e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:00.017961 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e5d26-37f6-4faf-a870-3c8009bbca20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:00.018819 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e5d26-37f6-4faf-a870-3c8009bbca20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:00.019749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e5d26-37f6-4faf-a870-3c8009bbca20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:00.020381 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e5d26-37f6-4faf-a870-3c8009bbca20 admin admin] PATH_INFO: `/v3/projects/baa2d28aea334dc9a029f30ec7799286` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:00.022267 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e5d26-37f6-4faf-a870-3c8009bbca20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baa2d28aea334dc9a029f30ec7799286)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:00.023580 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-6e9e5d26-37f6-4faf-a870-3c8009bbca20 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:11:00.023580 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Aug 02 09:11:00.023580 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: Aug 02 09:11:00.025300 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e5d26-37f6-4faf-a870-3c8009bbca20 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:00.128412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f0b05-94aa-4100-9c97-42700610a098 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:00.129529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f0b05-94aa-4100-9c97-42700610a098 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:00.129529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f0b05-94aa-4100-9c97-42700610a098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:00.167001 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6e9e5d26-37f6-4faf-a870-3c8009bbca20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baa2d28aea334dc9a029f30ec7799286', 'request_id': 'req-6e9e5d26-37f6-4faf-a870-3c8009bbca20'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:00.190622 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-ba4f0b05-94aa-4100-9c97-42700610a098 None None] MFA Rules not processed for user `5bcbbb365c124b1d9987c31315571409`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:00.192490 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ba4f0b05-94aa-4100-9c97-42700610a098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:00.201587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6e9e5d26-37f6-4faf-a870-3c8009bbca20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baa2d28aea334dc9a029f30ec7799286', 'request_id': 'req-6e9e5d26-37f6-4faf-a870-3c8009bbca20'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:00.223519 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6e9e5d26-37f6-4faf-a870-3c8009bbca20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baa2d28aea334dc9a029f30ec7799286', 'request_id': 'req-6e9e5d26-37f6-4faf-a870-3c8009bbca20'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:00.243646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6e9e5d26-37f6-4faf-a870-3c8009bbca20 admin admin] The token cache is being invalidate because project baa2d28aea334dc9a029f30ec7799286 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:00.244200 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6e9e5d26-37f6-4faf-a870-3c8009bbca20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:00.245263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6e9e5d26-37f6-4faf-a870-3c8009bbca20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:00.247031 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 827/1653] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:10:59 2022] DELETE /identity/v3/projects/baa2d28aea334dc9a029f30ec7799286 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:00.253155 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e5d26-37f6-4faf-a870-3c8009bbca20 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:00.266651 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6e9e5d26-37f6-4faf-a870-3c8009bbca20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:00.415753 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 827/1654] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:00 2022] POST /identity/v3/auth/tokens => generated 3357 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:11:00.417494 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f0b05-94aa-4100-9c97-42700610a098 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:00.433027 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ba4f0b05-94aa-4100-9c97-42700610a098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:00.490429 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e5d26-37f6-4faf-a870-3c8009bbca20 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:00.519159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5906167-2882-426f-b881-51f9330099bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:00.520410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5906167-2882-426f-b881-51f9330099bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:00.520692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5906167-2882-426f-b881-51f9330099bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:00.520972 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5906167-2882-426f-b881-51f9330099bd admin admin] PATH_INFO: `/v3/users/529ee7f2c9844888a5bb68108aec4c1a` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:00.530125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5906167-2882-426f-b881-51f9330099bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=529ee7f2c9844888a5bb68108aec4c1a)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:00.532870 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5906167-2882-426f-b881-51f9330099bd admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:00.680584 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c5906167-2882-426f-b881-51f9330099bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '529ee7f2c9844888a5bb68108aec4c1a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:00.705841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c5906167-2882-426f-b881-51f9330099bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '529ee7f2c9844888a5bb68108aec4c1a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:00.706625 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c5906167-2882-426f-b881-51f9330099bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '529ee7f2c9844888a5bb68108aec4c1a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:00.732241 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c5906167-2882-426f-b881-51f9330099bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '529ee7f2c9844888a5bb68108aec4c1a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:00.732835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c5906167-2882-426f-b881-51f9330099bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '529ee7f2c9844888a5bb68108aec4c1a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:00.748211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f0b05-94aa-4100-9c97-42700610a098 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:00.772116 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08c67c-49c1-4589-b9b1-ff5396992d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:00.772922 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08c67c-49c1-4589-b9b1-ff5396992d7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:00.773338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08c67c-49c1-4589-b9b1-ff5396992d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:00.773690 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08c67c-49c1-4589-b9b1-ff5396992d7d admin admin] PATH_INFO: `/v3/projects/e44014cd67e24815b6783a7cebacc760` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:00.780782 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 828/1655] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:11:00 2022] DELETE /identity/v3/users/529ee7f2c9844888a5bb68108aec4c1a => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:00.796163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c08c67c-49c1-4589-b9b1-ff5396992d7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e44014cd67e24815b6783a7cebacc760)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:00.810950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c08c67c-49c1-4589-b9b1-ff5396992d7d admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:00.991637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-0c08c67c-49c1-4589-b9b1-ff5396992d7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e44014cd67e24815b6783a7cebacc760', 'request_id': 'req-0c08c67c-49c1-4589-b9b1-ff5396992d7d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:01.014470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-0c08c67c-49c1-4589-b9b1-ff5396992d7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e44014cd67e24815b6783a7cebacc760', 'request_id': 'req-0c08c67c-49c1-4589-b9b1-ff5396992d7d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:01.041172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-0c08c67c-49c1-4589-b9b1-ff5396992d7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e44014cd67e24815b6783a7cebacc760', 'request_id': 'req-0c08c67c-49c1-4589-b9b1-ff5396992d7d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:01.070479 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-0c08c67c-49c1-4589-b9b1-ff5396992d7d admin admin] The token cache is being invalidate because project e44014cd67e24815b6783a7cebacc760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:01.070809 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-0c08c67c-49c1-4589-b9b1-ff5396992d7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:01.072924 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-0c08c67c-49c1-4589-b9b1-ff5396992d7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:01.081864 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 828/1656] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:11:00 2022] DELETE /identity/v3/projects/e44014cd67e24815b6783a7cebacc760 => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:01.083944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6850c-0ba5-44cd-80f5-ff9828766417 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:01.084483 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6850c-0ba5-44cd-80f5-ff9828766417 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:01.084734 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6850c-0ba5-44cd-80f5-ff9828766417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:01.099792 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:01.141671 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f4c6850c-0ba5-44cd-80f5-ff9828766417 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:01.142732 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f4c6850c-0ba5-44cd-80f5-ff9828766417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:01.419087 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 829/1657] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:01.431136 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba027793-b10d-4f44-b650-585f518ca535 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:01.432002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba027793-b10d-4f44-b650-585f518ca535 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:01.432434 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba027793-b10d-4f44-b650-585f518ca535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:01.478618 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-ba027793-b10d-4f44-b650-585f518ca535 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:01.480069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ba027793-b10d-4f44-b650-585f518ca535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:01.504172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 829/1658] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:01.509417 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6850c-0ba5-44cd-80f5-ff9828766417 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:01.527961 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6850c-0ba5-44cd-80f5-ff9828766417 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:01.546919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31cbf3-c0cc-4e35-a177-9ad236a6574d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:01.548879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31cbf3-c0cc-4e35-a177-9ad236a6574d admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:01.549201 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31cbf3-c0cc-4e35-a177-9ad236a6574d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:01.549487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31cbf3-c0cc-4e35-a177-9ad236a6574d admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:01.553713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31cbf3-c0cc-4e35-a177-9ad236a6574d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:11:01.554066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31cbf3-c0cc-4e35-a177-9ad236a6574d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:01.555724 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31cbf3-c0cc-4e35-a177-9ad236a6574d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:01.580410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 830/1659] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:11:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:11:01.585267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba027793-b10d-4f44-b650-585f518ca535 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:01.592836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:01.596882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba027793-b10d-4f44-b650-585f518ca535 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:01.615591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a60ff-449d-4b92-a8de-54d2732e90eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:01.616243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a60ff-449d-4b92-a8de-54d2732e90eb admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:01.616545 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a60ff-449d-4b92-a8de-54d2732e90eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:01.616955 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a60ff-449d-4b92-a8de-54d2732e90eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:01.617672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a60ff-449d-4b92-a8de-54d2732e90eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:01.632191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a60ff-449d-4b92-a8de-54d2732e90eb admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:01.694514 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 830/1660] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:11:01 2022] POST /identity/v3/projects => generated 366 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:01.698603 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31cbf3-c0cc-4e35-a177-9ad236a6574d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:01.717435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31cbf3-c0cc-4e35-a177-9ad236a6574d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:01.893913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a32654-c39e-4f47-8fea-2e7c830221fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:01.895203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a32654-c39e-4f47-8fea-2e7c830221fe admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:01.895203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a32654-c39e-4f47-8fea-2e7c830221fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:01.900587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a32654-c39e-4f47-8fea-2e7c830221fe admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:01.911808 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a32654-c39e-4f47-8fea-2e7c830221fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:01.913567 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a32654-c39e-4f47-8fea-2e7c830221fe admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:01.997941 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 831/1661] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:11:01 2022] POST /identity/v3/users => generated 284 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:02.002164 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a60ff-449d-4b92-a8de-54d2732e90eb admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:02.019756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a60ff-449d-4b92-a8de-54d2732e90eb admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:02.045890 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3838d9-0031-4342-8c36-e4219ba4c4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:02.048280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3838d9-0031-4342-8c36-e4219ba4c4d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:02.049230 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3838d9-0031-4342-8c36-e4219ba4c4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:02.049525 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3838d9-0031-4342-8c36-e4219ba4c4d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:02.050281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3838d9-0031-4342-8c36-e4219ba4c4d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:02.061998 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3838d9-0031-4342-8c36-e4219ba4c4d8 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:02.112121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 831/1662] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:02 2022] GET /identity/v3/roles => generated 1427 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:02.113236 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a32654-c39e-4f47-8fea-2e7c830221fe admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:02.125226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a32654-c39e-4f47-8fea-2e7c830221fe admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:02.147458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5424d67-82d9-4ffc-adcc-5f7bd0565f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:02.149549 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5424d67-82d9-4ffc-adcc-5f7bd0565f8c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:02.150093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5424d67-82d9-4ffc-adcc-5f7bd0565f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:02.150570 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5424d67-82d9-4ffc-adcc-5f7bd0565f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:02.151506 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5424d67-82d9-4ffc-adcc-5f7bd0565f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:02.154073 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5424d67-82d9-4ffc-adcc-5f7bd0565f8c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:02.191867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 832/1663] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:02 2022] GET /identity/v3/roles => generated 1427 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:02.197747 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3838d9-0031-4342-8c36-e4219ba4c4d8 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:02.220202 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3838d9-0031-4342-8c36-e4219ba4c4d8 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:02.237434 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5947c2f-0a0d-4b39-98a4-38ede05eb032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:02.241013 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5947c2f-0a0d-4b39-98a4-38ede05eb032 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:02.241330 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5947c2f-0a0d-4b39-98a4-38ede05eb032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:02.247080 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5947c2f-0a0d-4b39-98a4-38ede05eb032 admin admin] PATH_INFO: `/v3/projects/71f3d2361688490ea2edc2c1e6ca7664/users/f2cbb48facb3451b853c4dcb44b20cd2/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:02.285372 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5947c2f-0a0d-4b39-98a4-38ede05eb032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71f3d2361688490ea2edc2c1e6ca7664, user_id=f2cbb48facb3451b853c4dcb44b20cd2, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:02.290296 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5947c2f-0a0d-4b39-98a4-38ede05eb032 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:02.356680 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77222e6f-90cf-4386-8866-d0abed447933 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:02.357365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77222e6f-90cf-4386-8866-d0abed447933 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:02.357731 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 832/1664] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:02 2022] PUT /identity/v3/projects/71f3d2361688490ea2edc2c1e6ca7664/users/f2cbb48facb3451b853c4dcb44b20cd2/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:02.358025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77222e6f-90cf-4386-8866-d0abed447933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:02.402010 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5947c2f-0a0d-4b39-98a4-38ede05eb032 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:02.404207 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f5947c2f-0a0d-4b39-98a4-38ede05eb032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:02.418023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-77222e6f-90cf-4386-8866-d0abed447933 None None] MFA Rules not processed for user `f2cbb48facb3451b853c4dcb44b20cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:02.419893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-77222e6f-90cf-4386-8866-d0abed447933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:02.582638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:02.608073 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 833/1665] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:11:02.609622 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77222e6f-90cf-4386-8866-d0abed447933 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:02.631569 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5947c2f-0a0d-4b39-98a4-38ede05eb032 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:02.646399 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77222e6f-90cf-4386-8866-d0abed447933 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:02.649942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f3c8d-86d2-4a55-97f9-05b30efb6979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:02.650630 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f3c8d-86d2-4a55-97f9-05b30efb6979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:02.650915 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f3c8d-86d2-4a55-97f9-05b30efb6979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:02.651199 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f3c8d-86d2-4a55-97f9-05b30efb6979 admin admin] PATH_INFO: `/v3/projects/78e20dede1f648e4ac0dd0ecca6665b9` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:02.652504 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f3c8d-86d2-4a55-97f9-05b30efb6979 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78e20dede1f648e4ac0dd0ecca6665b9)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:02.654682 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f3c8d-86d2-4a55-97f9-05b30efb6979 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:02.667777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614eeedf-325b-40a0-8bde-a286fe62ede0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:02.668704 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614eeedf-325b-40a0-8bde-a286fe62ede0 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:02.669149 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614eeedf-325b-40a0-8bde-a286fe62ede0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:02.669559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614eeedf-325b-40a0-8bde-a286fe62ede0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:02.670553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614eeedf-325b-40a0-8bde-a286fe62ede0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:02.675279 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614eeedf-325b-40a0-8bde-a286fe62ede0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:02.740530 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 834/1666] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:11:02 2022] POST /identity/v3/projects => generated 364 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:02.744937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614eeedf-325b-40a0-8bde-a286fe62ede0 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:02.850688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-2c1f3c8d-86d2-4a55-97f9-05b30efb6979 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78e20dede1f648e4ac0dd0ecca6665b9', 'request_id': 'req-2c1f3c8d-86d2-4a55-97f9-05b30efb6979'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:02.865361 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-2c1f3c8d-86d2-4a55-97f9-05b30efb6979 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78e20dede1f648e4ac0dd0ecca6665b9', 'request_id': 'req-2c1f3c8d-86d2-4a55-97f9-05b30efb6979'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:02.889478 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-2c1f3c8d-86d2-4a55-97f9-05b30efb6979 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78e20dede1f648e4ac0dd0ecca6665b9', 'request_id': 'req-2c1f3c8d-86d2-4a55-97f9-05b30efb6979'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:02.951460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-2c1f3c8d-86d2-4a55-97f9-05b30efb6979 admin admin] The token cache is being invalidate because project 78e20dede1f648e4ac0dd0ecca6665b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:02.951769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-2c1f3c8d-86d2-4a55-97f9-05b30efb6979 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:02.960589 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-2c1f3c8d-86d2-4a55-97f9-05b30efb6979 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:02.962189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614eeedf-325b-40a0-8bde-a286fe62ede0 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:02.973865 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 833/1667] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:11:02 2022] DELETE /identity/v3/projects/78e20dede1f648e4ac0dd0ecca6665b9 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:02.977247 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521a7ec-11ed-46cf-8e05-f1326f844477 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:02.977552 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521a7ec-11ed-46cf-8e05-f1326f844477 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:02.977836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521a7ec-11ed-46cf-8e05-f1326f844477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:03.065618 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-1521a7ec-11ed-46cf-8e05-f1326f844477 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:03.068104 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1521a7ec-11ed-46cf-8e05-f1326f844477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:03.147434 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:03.183166 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631d261-6e98-4578-83bb-987b4d1e73b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:03.183824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631d261-6e98-4578-83bb-987b4d1e73b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:03.184134 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631d261-6e98-4578-83bb-987b4d1e73b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:03.184408 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631d261-6e98-4578-83bb-987b4d1e73b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:03.185086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631d261-6e98-4578-83bb-987b4d1e73b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:03.187070 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631d261-6e98-4578-83bb-987b4d1e73b0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:03.212887 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 834/1668] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:03.214067 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521a7ec-11ed-46cf-8e05-f1326f844477 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:03.221701 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1521a7ec-11ed-46cf-8e05-f1326f844477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:03.243290 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 835/1669] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:11:02 2022] POST /identity/v3/users => generated 289 bytes in 496 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:03.243290 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a521be-04ff-447a-9486-427e34270374 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:03.243290 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a521be-04ff-447a-9486-427e34270374 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:03.243290 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a521be-04ff-447a-9486-427e34270374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:03.266434 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521a7ec-11ed-46cf-8e05-f1326f844477 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:03.278690 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd15b35-2183-4832-8735-e8983063234d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:03.283117 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd15b35-2183-4832-8735-e8983063234d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:03.283860 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd15b35-2183-4832-8735-e8983063234d admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:03.284386 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd15b35-2183-4832-8735-e8983063234d admin admin] PATH_INFO: `/v3/projects/0d3b7509951d4beeacd7f741e7db83ba` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:03.287821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd15b35-2183-4832-8735-e8983063234d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d3b7509951d4beeacd7f741e7db83ba)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:03.290321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd15b35-2183-4832-8735-e8983063234d admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:03.303969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c3a521be-04ff-447a-9486-427e34270374 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:03.305684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c3a521be-04ff-447a-9486-427e34270374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:03.349891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 836/1670] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:03 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:03.349891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a521be-04ff-447a-9486-427e34270374 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:03.349891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c3a521be-04ff-447a-9486-427e34270374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:03.371199 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-6cd15b35-2183-4832-8735-e8983063234d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d3b7509951d4beeacd7f741e7db83ba', 'request_id': 'req-6cd15b35-2183-4832-8735-e8983063234d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:03.379710 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a521be-04ff-447a-9486-427e34270374 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:03.389632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-6cd15b35-2183-4832-8735-e8983063234d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d3b7509951d4beeacd7f741e7db83ba', 'request_id': 'req-6cd15b35-2183-4832-8735-e8983063234d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:03.419618 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-6cd15b35-2183-4832-8735-e8983063234d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d3b7509951d4beeacd7f741e7db83ba', 'request_id': 'req-6cd15b35-2183-4832-8735-e8983063234d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:03.434816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-6cd15b35-2183-4832-8735-e8983063234d admin admin] The token cache is being invalidate because project 0d3b7509951d4beeacd7f741e7db83ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:03.435274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-6cd15b35-2183-4832-8735-e8983063234d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:03.436075 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-6cd15b35-2183-4832-8735-e8983063234d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:03.438619 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 835/1671] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:11:03 2022] DELETE /identity/v3/projects/0d3b7509951d4beeacd7f741e7db83ba => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:03.439344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd15b35-2183-4832-8735-e8983063234d admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:03.441156 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-6cd15b35-2183-4832-8735-e8983063234d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:03.546019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedd385-c462-4a41-9cd7-9162dbdc8f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:03.547424 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedd385-c462-4a41-9cd7-9162dbdc8f5a admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:03.547885 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedd385-c462-4a41-9cd7-9162dbdc8f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:03.548491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedd385-c462-4a41-9cd7-9162dbdc8f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:03.549900 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedd385-c462-4a41-9cd7-9162dbdc8f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:03.554922 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd15b35-2183-4832-8735-e8983063234d admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:03.555336 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedd385-c462-4a41-9cd7-9162dbdc8f5a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:03.561372 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be9a30-f5c5-4f5d-89b2-da768303fd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:03.562452 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be9a30-f5c5-4f5d-89b2-da768303fd55 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:03.562763 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be9a30-f5c5-4f5d-89b2-da768303fd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:03.563047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be9a30-f5c5-4f5d-89b2-da768303fd55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:03.564044 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be9a30-f5c5-4f5d-89b2-da768303fd55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:11:03.564367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be9a30-f5c5-4f5d-89b2-da768303fd55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:03.566382 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be9a30-f5c5-4f5d-89b2-da768303fd55 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:03.572025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 837/1672] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:03 2022] GET /identity/v3/roles => generated 1427 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:03.573634 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3245e0-900d-459b-afbe-41a21f5f05af None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:03.574130 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3245e0-900d-459b-afbe-41a21f5f05af None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:03.574586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3245e0-900d-459b-afbe-41a21f5f05af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:03.587015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:03.606034 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 836/1673] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:11:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:11:03.606034 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be9a30-f5c5-4f5d-89b2-da768303fd55 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:03.607007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-16be9a30-f5c5-4f5d-89b2-da768303fd55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:03.613598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-7c3245e0-900d-459b-afbe-41a21f5f05af None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:03.615385 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7c3245e0-900d-459b-afbe-41a21f5f05af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:03.625161 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 838/1674] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:03 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:03.626352 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3245e0-900d-459b-afbe-41a21f5f05af None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:03.631700 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be9a30-f5c5-4f5d-89b2-da768303fd55 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:03.638888 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3245e0-900d-459b-afbe-41a21f5f05af None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:03.646826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb125c-216d-4009-9b8a-34bf85b42c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:03.647214 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351afa2-fa00-4243-891a-62085ff56e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:03.648106 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351afa2-fa00-4243-891a-62085ff56e31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:03.648538 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351afa2-fa00-4243-891a-62085ff56e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:03.648948 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351afa2-fa00-4243-891a-62085ff56e31 admin admin] PATH_INFO: `/v3/projects/f2dd1fe506ae41c5a8192bdee184ee44/users/19a046e442ad4affa5837c10bc1077d1/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:03.661920 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb125c-216d-4009-9b8a-34bf85b42c41 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:03.662235 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb125c-216d-4009-9b8a-34bf85b42c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:03.663239 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb125c-216d-4009-9b8a-34bf85b42c41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:03.665831 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb125c-216d-4009-9b8a-34bf85b42c41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:03.681474 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb125c-216d-4009-9b8a-34bf85b42c41 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:03.682612 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351afa2-fa00-4243-891a-62085ff56e31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2dd1fe506ae41c5a8192bdee184ee44, user_id=19a046e442ad4affa5837c10bc1077d1, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:03.687341 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351afa2-fa00-4243-891a-62085ff56e31 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:03.716154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 837/1675] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:03 2022] PUT /identity/v3/projects/f2dd1fe506ae41c5a8192bdee184ee44/users/19a046e442ad4affa5837c10bc1077d1/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:03.717827 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98816721-e4a3-4f57-944a-85517a7116da None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:03.719175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98816721-e4a3-4f57-944a-85517a7116da None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:03.723571 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98816721-e4a3-4f57-944a-85517a7116da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:03.752125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 839/1676] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:11:03 2022] POST /identity/v3/projects => generated 386 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:03.753971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb125c-216d-4009-9b8a-34bf85b42c41 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:03.787301 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-98816721-e4a3-4f57-944a-85517a7116da None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:03.790788 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-98816721-e4a3-4f57-944a-85517a7116da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:03.843964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:03.918827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb125c-216d-4009-9b8a-34bf85b42c41 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:03.932428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f555d2-d62c-4e3d-bda2-21e5d84ab6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:03.933177 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f555d2-d62c-4e3d-bda2-21e5d84ab6f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:03.933526 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f555d2-d62c-4e3d-bda2-21e5d84ab6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:03.933866 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f555d2-d62c-4e3d-bda2-21e5d84ab6f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:03.934640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f555d2-d62c-4e3d-bda2-21e5d84ab6f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:03.936405 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f555d2-d62c-4e3d-bda2-21e5d84ab6f1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:03.949390 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 838/1677] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:03 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:03.953044 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98816721-e4a3-4f57-944a-85517a7116da None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:03.964865 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98816721-e4a3-4f57-944a-85517a7116da None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:03.979604 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 840/1678] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:03 2022] GET /identity/v3/roles => generated 1427 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:03.980532 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f555d2-d62c-4e3d-bda2-21e5d84ab6f1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:03.981746 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad2c32-fe96-4c6e-8bda-d0c3c3a59398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:03.982394 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad2c32-fe96-4c6e-8bda-d0c3c3a59398 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:03.982671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad2c32-fe96-4c6e-8bda-d0c3c3a59398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:03.982955 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad2c32-fe96-4c6e-8bda-d0c3c3a59398 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:03.983848 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad2c32-fe96-4c6e-8bda-d0c3c3a59398 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:03.986503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad2c32-fe96-4c6e-8bda-d0c3c3a59398 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:04.015673 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f555d2-d62c-4e3d-bda2-21e5d84ab6f1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:04.026547 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ad23d-456c-45d3-91f7-7115de4ed67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:04.027943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ad23d-456c-45d3-91f7-7115de4ed67e admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:04.028262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ad23d-456c-45d3-91f7-7115de4ed67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:04.033341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ad23d-456c-45d3-91f7-7115de4ed67e admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:04.036193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ad23d-456c-45d3-91f7-7115de4ed67e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:11:04.036559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ad23d-456c-45d3-91f7-7115de4ed67e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:04.040242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ad23d-456c-45d3-91f7-7115de4ed67e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:04.047890 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 839/1679] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:11:03 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:04.049232 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83b259-49a8-4f03-91bb-56158fcadbeb None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:04.051953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83b259-49a8-4f03-91bb-56158fcadbeb None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:04.052261 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83b259-49a8-4f03-91bb-56158fcadbeb None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:04.056020 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 840/1680] 10.222.0.100 () {54 vars in 926 bytes} [Tue Aug 2 09:11:04 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:11:04.057152 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83b259-49a8-4f03-91bb-56158fcadbeb None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:04.080933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 841/1681] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:11:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:11:04.082971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ad23d-456c-45d3-91f7-7115de4ed67e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:04.093076 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83b259-49a8-4f03-91bb-56158fcadbeb None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:04.101528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0776115-8673-42cc-89ca-b7cc8e871338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:04.101999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0776115-8673-42cc-89ca-b7cc8e871338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:04.102286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0776115-8673-42cc-89ca-b7cc8e871338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:04.102569 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0776115-8673-42cc-89ca-b7cc8e871338 admin admin] PATH_INFO: `/v3/projects/f2dd1fe506ae41c5a8192bdee184ee44/users/19a046e442ad4affa5837c10bc1077d1/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:04.105859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0776115-8673-42cc-89ca-b7cc8e871338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2dd1fe506ae41c5a8192bdee184ee44, user_id=19a046e442ad4affa5837c10bc1077d1, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:04.109352 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0776115-8673-42cc-89ca-b7cc8e871338 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:04.109492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ad23d-456c-45d3-91f7-7115de4ed67e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:04.127225 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb22b0-1418-4d50-a6c7-dc29d73ca173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:04.130305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb22b0-1418-4d50-a6c7-dc29d73ca173 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:04.133606 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 841/1682] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:04 2022] PUT /identity/v3/projects/f2dd1fe506ae41c5a8192bdee184ee44/users/19a046e442ad4affa5837c10bc1077d1/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:04.146142 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb22b0-1418-4d50-a6c7-dc29d73ca173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:04.146734 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb22b0-1418-4d50-a6c7-dc29d73ca173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:04.147747 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb22b0-1418-4d50-a6c7-dc29d73ca173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:04.150592 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0776115-8673-42cc-89ca-b7cc8e871338 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:04.151640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb22b0-1418-4d50-a6c7-dc29d73ca173 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:04.174982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 842/1683] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:04 2022] GET /identity/v3/roles => generated 1427 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:04.177352 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb151047-ad70-4b4c-9b62-0e3839e7cc37 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:04.178142 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb151047-ad70-4b4c-9b62-0e3839e7cc37 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:04.178524 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb151047-ad70-4b4c-9b62-0e3839e7cc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:04.212815 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-eb151047-ad70-4b4c-9b62-0e3839e7cc37 None None] MFA Rules not processed for user `b40981fe179a43dc81553d3594dad649`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:04.215486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-eb151047-ad70-4b4c-9b62-0e3839e7cc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:04.308348 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0776115-8673-42cc-89ca-b7cc8e871338 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:04.315127 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5bfc2-904b-4ff3-b325-b7be7f11af0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:04.316609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5bfc2-904b-4ff3-b325-b7be7f11af0e admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:04.317451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5bfc2-904b-4ff3-b325-b7be7f11af0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:04.319147 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5bfc2-904b-4ff3-b325-b7be7f11af0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:04.320164 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5bfc2-904b-4ff3-b325-b7be7f11af0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:04.322558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5bfc2-904b-4ff3-b325-b7be7f11af0e admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:04.432127 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 842/1684] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:11:04 2022] POST /identity/v3/projects => generated 372 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:04.432127 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d48c0-41f0-4cda-be88-813c505826a0 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:04.432127 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d48c0-41f0-4cda-be88-813c505826a0 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:04.432127 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d48c0-41f0-4cda-be88-813c505826a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:04.432822 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 843/1685] 10.222.0.100 () {58 vars in 1023 bytes} [Tue Aug 2 09:11:04 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:04.432822 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb151047-ad70-4b4c-9b62-0e3839e7cc37 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:04.435940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb151047-ad70-4b4c-9b62-0e3839e7cc37 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:04.463971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-7e2d48c0-41f0-4cda-be88-813c505826a0 None None] MFA Rules not processed for user `19a046e442ad4affa5837c10bc1077d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:04.466743 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-7e2d48c0-41f0-4cda-be88-813c505826a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:04.542263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:04.610726 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2688d53-3c12-49ea-8c0f-51db935a9472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:04.611392 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2688d53-3c12-49ea-8c0f-51db935a9472 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:04.611681 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2688d53-3c12-49ea-8c0f-51db935a9472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:04.611965 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2688d53-3c12-49ea-8c0f-51db935a9472 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:04.612686 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2688d53-3c12-49ea-8c0f-51db935a9472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:04.614946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2688d53-3c12-49ea-8c0f-51db935a9472 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:04.621035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 843/1686] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:04.623214 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709dc1bd-fe0f-45f2-b9ca-fc9de96bd52c None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:04.631282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709dc1bd-fe0f-45f2-b9ca-fc9de96bd52c None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:04.632758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709dc1bd-fe0f-45f2-b9ca-fc9de96bd52c None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:04.635462 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 844/1687] 10.222.0.100 () {54 vars in 966 bytes} [Tue Aug 2 09:11:04 2022] GET /identity => generated 271 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:11:04.642365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709dc1bd-fe0f-45f2-b9ca-fc9de96bd52c None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:04.644867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 844/1688] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:04 2022] GET /identity/v3/roles => generated 1427 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:04.646428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f02e82-a037-4e92-9c5f-591549d7e7a2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:04.646734 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f02e82-a037-4e92-9c5f-591549d7e7a2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:04.647096 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f02e82-a037-4e92-9c5f-591549d7e7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:04.712759 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-15f02e82-a037-4e92-9c5f-591549d7e7a2 None None] MFA Rules not processed for user `c2782f4783624ca39ace71143b3b9237`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:04.714656 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-15f02e82-a037-4e92-9c5f-591549d7e7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:04.731567 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709dc1bd-fe0f-45f2-b9ca-fc9de96bd52c None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:04.752767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a1f9a-b562-4172-b0d0-d8d7e51157d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:04.755435 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a1f9a-b562-4172-b0d0-d8d7e51157d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:04.755732 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a1f9a-b562-4172-b0d0-d8d7e51157d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:04.758428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a1f9a-b562-4172-b0d0-d8d7e51157d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:04.763415 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a1f9a-b562-4172-b0d0-d8d7e51157d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:04.772670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a1f9a-b562-4172-b0d0-d8d7e51157d4 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:04.833737 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:04.875864 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 845/1689] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:11:04 2022] POST /identity/v3/users => generated 287 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:04.878551 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f5071-0ddc-4c2b-8226-c09b166152a4 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:04.879661 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f5071-0ddc-4c2b-8226-c09b166152a4 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:04.880493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f5071-0ddc-4c2b-8226-c09b166152a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:04.883306 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 845/1690] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:04.895647 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f02e82-a037-4e92-9c5f-591549d7e7a2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:04.900765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-15f02e82-a037-4e92-9c5f-591549d7e7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:04.958414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-f88f5071-0ddc-4c2b-8226-c09b166152a4 None None] MFA Rules not processed for user `b40981fe179a43dc81553d3594dad649`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:04.960933 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f88f5071-0ddc-4c2b-8226-c09b166152a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:05.023945 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f02e82-a037-4e92-9c5f-591549d7e7a2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:05.031067 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25b6c7-e04c-4457-9e70-735dc90a5751 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:05.031719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25b6c7-e04c-4457-9e70-735dc90a5751 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:05.032006 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25b6c7-e04c-4457-9e70-735dc90a5751 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:05.032288 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25b6c7-e04c-4457-9e70-735dc90a5751 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:05.058292 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f25b6c7-e04c-4457-9e70-735dc90a5751 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:05.059803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f25b6c7-e04c-4457-9e70-735dc90a5751 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:05.080411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 846/1691] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:05.080698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25b6c7-e04c-4457-9e70-735dc90a5751 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:05.102722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25b6c7-e04c-4457-9e70-735dc90a5751 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:05.119224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcfc63f-569f-48c3-8610-22646f72c7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:05.119937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfc63f-569f-48c3-8610-22646f72c7c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:05.120228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfc63f-569f-48c3-8610-22646f72c7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:05.120512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfc63f-569f-48c3-8610-22646f72c7c6 admin admin] PATH_INFO: `/v3/projects/2d8d7b19037b4a1b95bfbf97fd624645/users/911209f902d24b35b774169b62e2a5f6/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:05.121988 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 846/1692] 10.222.0.100 () {58 vars in 1063 bytes} [Tue Aug 2 09:11:04 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:05.122950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f5071-0ddc-4c2b-8226-c09b166152a4 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:05.157053 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f5071-0ddc-4c2b-8226-c09b166152a4 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:05.162353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da5f58-5672-458c-a758-b70bfd9538b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:05.163137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da5f58-5672-458c-a758-b70bfd9538b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:05.163610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da5f58-5672-458c-a758-b70bfd9538b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:05.164020 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da5f58-5672-458c-a758-b70bfd9538b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:05.164920 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da5f58-5672-458c-a758-b70bfd9538b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:05.168371 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da5f58-5672-458c-a758-b70bfd9538b8 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:05.169489 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcfc63f-569f-48c3-8610-22646f72c7c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d8d7b19037b4a1b95bfbf97fd624645, user_id=911209f902d24b35b774169b62e2a5f6, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:05.172206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcfc63f-569f-48c3-8610-22646f72c7c6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:05.184443 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 847/1693] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:05 2022] GET /identity/v3/roles => generated 1427 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:05.190566 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d290f69-530a-4ec9-8926-0030e5c7283b None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:05.193143 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d290f69-530a-4ec9-8926-0030e5c7283b None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:05.193707 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d290f69-530a-4ec9-8926-0030e5c7283b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:05.203979 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 847/1694] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:05 2022] PUT /identity/v3/projects/2d8d7b19037b4a1b95bfbf97fd624645/users/911209f902d24b35b774169b62e2a5f6/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:05.213548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcfc63f-569f-48c3-8610-22646f72c7c6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:05.241714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-9d290f69-530a-4ec9-8926-0030e5c7283b None None] MFA Rules not processed for user `c2782f4783624ca39ace71143b3b9237`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:05.243210 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-9d290f69-530a-4ec9-8926-0030e5c7283b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:05.320893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:05.361837 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcfc63f-569f-48c3-8610-22646f72c7c6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:05.380405 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4770bd9f-c214-45c8-a805-5c62b3752506 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b40981fe179a43dc81553d3594dad649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:05.387037 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 848/1695] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:05.392186 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770bd9f-c214-45c8-a805-5c62b3752506 service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:05.392724 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770bd9f-c214-45c8-a805-5c62b3752506 service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:05.393089 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770bd9f-c214-45c8-a805-5c62b3752506 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:05.396056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d290f69-530a-4ec9-8926-0030e5c7283b None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:05.413386 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4770bd9f-c214-45c8-a805-5c62b3752506 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:05.419515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4770bd9f-c214-45c8-a805-5c62b3752506 service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:05.470692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 848/1696] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:05 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:05.476560 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967857c-ab9d-46cf-814b-e06a1f1e484d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:05.476560 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967857c-ab9d-46cf-814b-e06a1f1e484d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:05.483891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967857c-ab9d-46cf-814b-e06a1f1e484d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:05.547681 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f967857c-ab9d-46cf-814b-e06a1f1e484d None None] MFA Rules not processed for user `911209f902d24b35b774169b62e2a5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:05.549825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f967857c-ab9d-46cf-814b-e06a1f1e484d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:05.578415 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d290f69-530a-4ec9-8926-0030e5c7283b None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:05.594220 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00dc3b-4f1c-420e-9efe-656dc497c29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:05.594980 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00dc3b-4f1c-420e-9efe-656dc497c29b admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:05.595342 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00dc3b-4f1c-420e-9efe-656dc497c29b admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:05.595694 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00dc3b-4f1c-420e-9efe-656dc497c29b admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:05.596487 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f00dc3b-4f1c-420e-9efe-656dc497c29b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:05.601274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f00dc3b-4f1c-420e-9efe-656dc497c29b admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:05.640903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 849/1697] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:05 2022] GET /identity/v3/roles => generated 1427 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:05.642998 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00dc3b-4f1c-420e-9efe-656dc497c29b admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:05.694755 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:05.721997 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 849/1698] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:05 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:05.724930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967857c-ab9d-46cf-814b-e06a1f1e484d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:05.728701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f967857c-ab9d-46cf-814b-e06a1f1e484d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:05.770026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00dc3b-4f1c-420e-9efe-656dc497c29b admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:05.776545 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ad963-1de4-40d4-919c-0a9345982d52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:05.777973 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ad963-1de4-40d4-919c-0a9345982d52 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:05.778418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ad963-1de4-40d4-919c-0a9345982d52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:05.778709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ad963-1de4-40d4-919c-0a9345982d52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:05.781556 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-fa4ad963-1de4-40d4-919c-0a9345982d52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:05.855755 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967857c-ab9d-46cf-814b-e06a1f1e484d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:05.861734 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636b24a-7ccc-47b9-8b99-f1346a633718 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:05.862478 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636b24a-7ccc-47b9-8b99-f1346a633718 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:05.863648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636b24a-7ccc-47b9-8b99-f1346a633718 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:05.863648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636b24a-7ccc-47b9-8b99-f1346a633718 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:05.881340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636b24a-7ccc-47b9-8b99-f1346a633718 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:05.883971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636b24a-7ccc-47b9-8b99-f1346a633718 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:05.895079 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ad963-1de4-40d4-919c-0a9345982d52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:05.895759 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 850/1699] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:05 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:05.896691 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636b24a-7ccc-47b9-8b99-f1346a633718 service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:05.897821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ad963-1de4-40d4-919c-0a9345982d52 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:05.908048 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636b24a-7ccc-47b9-8b99-f1346a633718 service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:05.914365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd305a04-d358-4d60-b9c0-15867f00b1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:05.915153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd305a04-d358-4d60-b9c0-15867f00b1d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:05.915511 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd305a04-d358-4d60-b9c0-15867f00b1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:05.915864 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd305a04-d358-4d60-b9c0-15867f00b1d1 admin admin] PATH_INFO: `/v3/projects/ec82b9b11f544b70977cf477be932c91/users/84b7ddd2dcf14086bd855fa288b9451c/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:05.921019 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 850/1700] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:05.922367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ad963-1de4-40d4-919c-0a9345982d52 service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:05.937770 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ad963-1de4-40d4-919c-0a9345982d52 service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:05.945642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b4d7b-0aec-4dce-8fe6-82752e5f2e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:05.946284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b4d7b-0aec-4dce-8fe6-82752e5f2e50 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:05.946572 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b4d7b-0aec-4dce-8fe6-82752e5f2e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:05.946853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b4d7b-0aec-4dce-8fe6-82752e5f2e50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:05.947528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b4d7b-0aec-4dce-8fe6-82752e5f2e50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:05.949711 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b4d7b-0aec-4dce-8fe6-82752e5f2e50 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:05.955578 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd305a04-d358-4d60-b9c0-15867f00b1d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec82b9b11f544b70977cf477be932c91, user_id=84b7ddd2dcf14086bd855fa288b9451c, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:05.964323 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd305a04-d358-4d60-b9c0-15867f00b1d1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:06.001156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 851/1701] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:05 2022] PUT /identity/v3/projects/ec82b9b11f544b70977cf477be932c91/users/84b7ddd2dcf14086bd855fa288b9451c/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:06.001366 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 851/1702] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:11:05 2022] POST /identity/v3/projects => generated 386 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:06.004244 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd305a04-d358-4d60-b9c0-15867f00b1d1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:06.013409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7feae-a8a2-43ff-a579-e116e3dba848 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:06.014346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7feae-a8a2-43ff-a579-e116e3dba848 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:06.015287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7feae-a8a2-43ff-a579-e116e3dba848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:06.100771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:06.178399 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-72d7feae-a8a2-43ff-a579-e116e3dba848 None None] MFA Rules not processed for user `84b7ddd2dcf14086bd855fa288b9451c`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:06.179901 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-72d7feae-a8a2-43ff-a579-e116e3dba848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:06.195388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd305a04-d358-4d60-b9c0-15867f00b1d1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:06.350887 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc24179-9d5b-4b51-8e4c-68ff93ba0219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:06.351537 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc24179-9d5b-4b51-8e4c-68ff93ba0219 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:06.351804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc24179-9d5b-4b51-8e4c-68ff93ba0219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:06.352082 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc24179-9d5b-4b51-8e4c-68ff93ba0219 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:06.352759 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc24179-9d5b-4b51-8e4c-68ff93ba0219 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:06.358540 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc24179-9d5b-4b51-8e4c-68ff93ba0219 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:06.414043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 852/1703] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:11:06.415338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7feae-a8a2-43ff-a579-e116e3dba848 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:06.427344 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 852/1704] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:11:06 2022] POST /identity/v3/users => generated 300 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:06.428254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc24179-9d5b-4b51-8e4c-68ff93ba0219 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:06.443134 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7feae-a8a2-43ff-a579-e116e3dba848 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:06.446293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc24179-9d5b-4b51-8e4c-68ff93ba0219 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:06.453174 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d21f5-f00b-48a8-bf76-3546002eebc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:06.453894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d21f5-f00b-48a8-bf76-3546002eebc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:06.454179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d21f5-f00b-48a8-bf76-3546002eebc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:06.454506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d21f5-f00b-48a8-bf76-3546002eebc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:06.455194 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d21f5-f00b-48a8-bf76-3546002eebc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:06.458134 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d21f5-f00b-48a8-bf76-3546002eebc0 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:06.461456 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62276ba1-8b36-412b-9055-5131cc6faadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:06.462141 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62276ba1-8b36-412b-9055-5131cc6faadf admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:06.462627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62276ba1-8b36-412b-9055-5131cc6faadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:06.462921 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62276ba1-8b36-412b-9055-5131cc6faadf admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:06.463691 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62276ba1-8b36-412b-9055-5131cc6faadf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:06.466269 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62276ba1-8b36-412b-9055-5131cc6faadf admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:06.511638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 853/1705] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:06 2022] GET /identity/v3/roles => generated 1427 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:06.512729 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62276ba1-8b36-412b-9055-5131cc6faadf admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:06.530280 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62276ba1-8b36-412b-9055-5131cc6faadf admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:06.547556 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 853/1706] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:11:06 2022] POST /identity/v3/projects => generated 374 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:11:06.547762 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba9594-07bd-43d2-885d-34f029630a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:06.549261 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d21f5-f00b-48a8-bf76-3546002eebc0 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:06.551434 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba9594-07bd-43d2-885d-34f029630a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:06.555981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba9594-07bd-43d2-885d-34f029630a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:06.556274 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba9594-07bd-43d2-885d-34f029630a2d admin admin] PATH_INFO: `/v3/projects/04e129d8b34941fba640d81a283f0773/users/1804fd3fa0854b90b17eb85b3dd1a59d/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:06.589472 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba9594-07bd-43d2-885d-34f029630a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=04e129d8b34941fba640d81a283f0773, user_id=1804fd3fa0854b90b17eb85b3dd1a59d, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:06.592401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba9594-07bd-43d2-885d-34f029630a2d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:06.592932 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:06.621494 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 854/1707] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:06 2022] PUT /identity/v3/projects/04e129d8b34941fba640d81a283f0773/users/1804fd3fa0854b90b17eb85b3dd1a59d/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:06.625655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba9594-07bd-43d2-885d-34f029630a2d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:06.689405 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d21f5-f00b-48a8-bf76-3546002eebc0 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:06.765795 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf5006-ce77-4e2b-8db3-23152c52bd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:06.766436 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf5006-ce77-4e2b-8db3-23152c52bd9c admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:06.766718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf5006-ce77-4e2b-8db3-23152c52bd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:06.766991 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf5006-ce77-4e2b-8db3-23152c52bd9c admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:06.767672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccf5006-ce77-4e2b-8db3-23152c52bd9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:06.770057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccf5006-ce77-4e2b-8db3-23152c52bd9c admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:06.783428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba9594-07bd-43d2-885d-34f029630a2d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:06.790597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e40deb-bbf7-41e3-859d-20e18ebfeccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:06.792401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e40deb-bbf7-41e3-859d-20e18ebfeccb admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:06.792746 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e40deb-bbf7-41e3-859d-20e18ebfeccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:06.794531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e40deb-bbf7-41e3-859d-20e18ebfeccb admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:06.795276 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e40deb-bbf7-41e3-859d-20e18ebfeccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:06.800009 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e40deb-bbf7-41e3-859d-20e18ebfeccb admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:06.812921 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 854/1708] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:11:06 2022] POST /identity/v3/users => generated 294 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:11:06.814367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf5006-ce77-4e2b-8db3-23152c52bd9c admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:06.832925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 855/1709] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:06 2022] GET /identity/v3/roles => generated 1427 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:06.837717 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e40deb-bbf7-41e3-859d-20e18ebfeccb admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:06.844663 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf5006-ce77-4e2b-8db3-23152c52bd9c admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:06.855078 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e40deb-bbf7-41e3-859d-20e18ebfeccb admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:06.857667 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e3961-4cf2-4654-b436-e9060046ba34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:06.858311 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e3961-4cf2-4654-b436-e9060046ba34 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:06.858691 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e3961-4cf2-4654-b436-e9060046ba34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:06.859166 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e3961-4cf2-4654-b436-e9060046ba34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:06.859906 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e3961-4cf2-4654-b436-e9060046ba34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:06.862351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e3961-4cf2-4654-b436-e9060046ba34 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:06.873073 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2037f-cc2a-423d-bdf7-c47ccaa16df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:06.873970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2037f-cc2a-423d-bdf7-c47ccaa16df1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:06.874252 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2037f-cc2a-423d-bdf7-c47ccaa16df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:06.874529 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2037f-cc2a-423d-bdf7-c47ccaa16df1 admin admin] PATH_INFO: `/v3/projects/04e129d8b34941fba640d81a283f0773/users/1804fd3fa0854b90b17eb85b3dd1a59d/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:06.882045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2037f-cc2a-423d-bdf7-c47ccaa16df1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04e129d8b34941fba640d81a283f0773, user_id=1804fd3fa0854b90b17eb85b3dd1a59d, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:06.888591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 855/1710] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:06 2022] GET /identity/v3/roles => generated 1427 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:06.892078 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2037f-cc2a-423d-bdf7-c47ccaa16df1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:06.894615 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e3961-4cf2-4654-b436-e9060046ba34 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:06.911497 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e3961-4cf2-4654-b436-e9060046ba34 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:06.920109 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 856/1711] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:06 2022] PUT /identity/v3/projects/04e129d8b34941fba640d81a283f0773/users/1804fd3fa0854b90b17eb85b3dd1a59d/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:06.932555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454fceb-095c-4378-9942-bbced4318e5c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:06.932845 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454fceb-095c-4378-9942-bbced4318e5c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:06.933150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454fceb-095c-4378-9942-bbced4318e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:06.984259 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:07.001203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9454fceb-095c-4378-9942-bbced4318e5c None None] MFA Rules not processed for user `1804fd3fa0854b90b17eb85b3dd1a59d`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:07.009666 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9454fceb-095c-4378-9942-bbced4318e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:07.027025 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f1814-c0b9-454b-8dee-fc336267b8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:07.030238 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f1814-c0b9-454b-8dee-fc336267b8a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:07.030238 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f1814-c0b9-454b-8dee-fc336267b8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:07.030238 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f1814-c0b9-454b-8dee-fc336267b8a4 admin admin] PATH_INFO: `/v3/projects/394f34a75f2d406382254e013f03838f/users/062391805ac4421ab679e69202b6702f/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:07.053917 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f1814-c0b9-454b-8dee-fc336267b8a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=394f34a75f2d406382254e013f03838f, user_id=062391805ac4421ab679e69202b6702f, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:07.056891 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f1814-c0b9-454b-8dee-fc336267b8a4 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:07.081157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 856/1712] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:06 2022] PUT /identity/v3/projects/394f34a75f2d406382254e013f03838f/users/062391805ac4421ab679e69202b6702f/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:07.081157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f1814-c0b9-454b-8dee-fc336267b8a4 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:07.081157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f46f1814-c0b9-454b-8dee-fc336267b8a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:07.118954 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:07.164524 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f1814-c0b9-454b-8dee-fc336267b8a4 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:07.169625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c75a5-1e74-4d74-8f33-84aa7dd24188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:07.170290 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c75a5-1e74-4d74-8f33-84aa7dd24188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:07.170570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c75a5-1e74-4d74-8f33-84aa7dd24188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:07.170845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c75a5-1e74-4d74-8f33-84aa7dd24188 admin admin] PATH_INFO: `/v3/users/cb07fc25243546ce87be2f456002147f` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:07.172073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c75a5-1e74-4d74-8f33-84aa7dd24188 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb07fc25243546ce87be2f456002147f)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:07.174942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c75a5-1e74-4d74-8f33-84aa7dd24188 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:07.226511 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 857/1713] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:06 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:07.227931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454fceb-095c-4378-9942-bbced4318e5c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:07.255727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454fceb-095c-4378-9942-bbced4318e5c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:07.286553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33476473-8be5-40bf-9463-a2082d9e5345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:07.288218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33476473-8be5-40bf-9463-a2082d9e5345 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:07.292596 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33476473-8be5-40bf-9463-a2082d9e5345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:07.293088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33476473-8be5-40bf-9463-a2082d9e5345 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:07.296705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33476473-8be5-40bf-9463-a2082d9e5345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:07.303498 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33476473-8be5-40bf-9463-a2082d9e5345 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:07.321260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d84c75a5-1e74-4d74-8f33-84aa7dd24188 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb07fc25243546ce87be2f456002147f'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:07.323440 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d84c75a5-1e74-4d74-8f33-84aa7dd24188 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb07fc25243546ce87be2f456002147f'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:07.338428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 858/1714] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:07 2022] GET /identity/v3/roles => generated 1427 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:07.340370 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33476473-8be5-40bf-9463-a2082d9e5345 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:07.353265 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d84c75a5-1e74-4d74-8f33-84aa7dd24188 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb07fc25243546ce87be2f456002147f'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:07.440207 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d84c75a5-1e74-4d74-8f33-84aa7dd24188 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb07fc25243546ce87be2f456002147f'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:07.466186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d84c75a5-1e74-4d74-8f33-84aa7dd24188 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb07fc25243546ce87be2f456002147f'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:07.493092 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 857/1715] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:11:07 2022] DELETE /identity/v3/users/cb07fc25243546ce87be2f456002147f => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:07.498132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b74c4-e244-4a46-9221-0cb1ed9db743 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:07.498583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b74c4-e244-4a46-9221-0cb1ed9db743 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:07.500475 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b74c4-e244-4a46-9221-0cb1ed9db743 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:07.508645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 858/1716] 10.222.0.100 () {54 vars in 917 bytes} [Tue Aug 2 09:11:07 2022] GET /identity => generated 271 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:11:07.514012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b74c4-e244-4a46-9221-0cb1ed9db743 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:07.520394 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-104b74c4-e244-4a46-9221-0cb1ed9db743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:07.594721 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33476473-8be5-40bf-9463-a2082d9e5345 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:07.809919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e82f6-eaf7-402a-ab8b-268d83171ce8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:07.810584 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e82f6-eaf7-402a-ab8b-268d83171ce8 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:07.810933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e82f6-eaf7-402a-ab8b-268d83171ce8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:07.811301 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b74c4-e244-4a46-9221-0cb1ed9db743 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:07.812118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e82f6-eaf7-402a-ab8b-268d83171ce8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:07.814129 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c60e82f6-eaf7-402a-ab8b-268d83171ce8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:07.821875 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079590b8-c189-4cd2-a25c-0941e288d374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:07.823660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079590b8-c189-4cd2-a25c-0941e288d374 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:07.824745 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079590b8-c189-4cd2-a25c-0941e288d374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:07.825729 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079590b8-c189-4cd2-a25c-0941e288d374 admin admin] PATH_INFO: `/v3/projects/394f34a75f2d406382254e013f03838f/users/062391805ac4421ab679e69202b6702f/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:07.834545 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079590b8-c189-4cd2-a25c-0941e288d374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=394f34a75f2d406382254e013f03838f, user_id=062391805ac4421ab679e69202b6702f, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:07.839803 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079590b8-c189-4cd2-a25c-0941e288d374 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:07.875703 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 859/1717] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:07 2022] PUT /identity/v3/projects/394f34a75f2d406382254e013f03838f/users/062391805ac4421ab679e69202b6702f/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:07.875703 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634c3e4-7d0a-43c0-9185-4acaf541ae96 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:07.875703 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634c3e4-7d0a-43c0-9185-4acaf541ae96 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:07.875703 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634c3e4-7d0a-43c0-9185-4acaf541ae96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:07.956325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:07.984050 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e82f6-eaf7-402a-ab8b-268d83171ce8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:07.985294 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-c634c3e4-7d0a-43c0-9185-4acaf541ae96 None None] MFA Rules not processed for user `f4afef33deb247509765a8c88a78ac39`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:07.985795 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e82f6-eaf7-402a-ab8b-268d83171ce8 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:07.986900 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c634c3e4-7d0a-43c0-9185-4acaf541ae96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:08.020438 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 859/1718] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:07 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 679 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:08.023395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0d636-7f06-4c7b-a2e8-71f2ab5e369a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:08.023708 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0d636-7f06-4c7b-a2e8-71f2ab5e369a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:08.023995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0d636-7f06-4c7b-a2e8-71f2ab5e369a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:08.071259 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:08.185411 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 860/1719] 10.222.0.100 () {58 vars in 1014 bytes} [Tue Aug 2 09:11:07 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:08.198415 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-fdb0d636-7f06-4c7b-a2e8-71f2ab5e369a None None] MFA Rules not processed for user `062391805ac4421ab679e69202b6702f`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:08.201664 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fdb0d636-7f06-4c7b-a2e8-71f2ab5e369a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:08.215626 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634c3e4-7d0a-43c0-9185-4acaf541ae96 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:08.217295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c634c3e4-7d0a-43c0-9185-4acaf541ae96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:08.418709 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 860/1720] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:08 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:08.419629 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0d636-7f06-4c7b-a2e8-71f2ab5e369a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:08.421445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fdb0d636-7f06-4c7b-a2e8-71f2ab5e369a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:08.462834 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0d636-7f06-4c7b-a2e8-71f2ab5e369a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:08.464165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634c3e4-7d0a-43c0-9185-4acaf541ae96 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:08.477991 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e262d-e654-426d-a8c7-b404afe0c0d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:08.478921 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e262d-e654-426d-a8c7-b404afe0c0d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:08.479375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e262d-e654-426d-a8c7-b404afe0c0d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:08.479814 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e262d-e654-426d-a8c7-b404afe0c0d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:08.480278 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633cba0d-be9b-4f1c-907f-9c92922ed840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:08.483569 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cba0d-be9b-4f1c-907f-9c92922ed840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:08.484017 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cba0d-be9b-4f1c-907f-9c92922ed840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:08.484539 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cba0d-be9b-4f1c-907f-9c92922ed840 admin admin] PATH_INFO: `/v3/users/911209f902d24b35b774169b62e2a5f6` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:08.494063 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e262d-e654-426d-a8c7-b404afe0c0d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:08.498979 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e262d-e654-426d-a8c7-b404afe0c0d5 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:08.500062 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633cba0d-be9b-4f1c-907f-9c92922ed840 admin admin] RBAC: Authorizing `identity:delete_user(user_id=911209f902d24b35b774169b62e2a5f6)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:08.502821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633cba0d-be9b-4f1c-907f-9c92922ed840 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:08.525338 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 861/1721] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:08 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:08.529322 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e262d-e654-426d-a8c7-b404afe0c0d5 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:08.533940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-630e262d-e654-426d-a8c7-b404afe0c0d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:08.658534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-633cba0d-be9b-4f1c-907f-9c92922ed840 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '911209f902d24b35b774169b62e2a5f6'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:08.660468 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-633cba0d-be9b-4f1c-907f-9c92922ed840 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '911209f902d24b35b774169b62e2a5f6'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:08.696401 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-633cba0d-be9b-4f1c-907f-9c92922ed840 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '911209f902d24b35b774169b62e2a5f6'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:08.736277 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e262d-e654-426d-a8c7-b404afe0c0d5 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:08.739781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-633cba0d-be9b-4f1c-907f-9c92922ed840 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '911209f902d24b35b774169b62e2a5f6'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:08.749821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05fba4-9330-442e-b7a2-b587921e8944 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:08.750786 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05fba4-9330-442e-b7a2-b587921e8944 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:08.751299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05fba4-9330-442e-b7a2-b587921e8944 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:08.751751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05fba4-9330-442e-b7a2-b587921e8944 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:08.786712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05fba4-9330-442e-b7a2-b587921e8944 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:08.788687 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05fba4-9330-442e-b7a2-b587921e8944 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:08.790739 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-633cba0d-be9b-4f1c-907f-9c92922ed840 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '911209f902d24b35b774169b62e2a5f6'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:08.816345 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 862/1722] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:08 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:08.818610 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05fba4-9330-442e-b7a2-b587921e8944 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:08.827852 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4d05fba4-9330-442e-b7a2-b587921e8944 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:08.837524 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 861/1723] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:11:08 2022] DELETE /identity/v3/users/911209f902d24b35b774169b62e2a5f6 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:11:08.843626 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e2873-56ee-44e1-9555-fbc9cec16a53 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:08.844037 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e2873-56ee-44e1-9555-fbc9cec16a53 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:08.846382 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e2873-56ee-44e1-9555-fbc9cec16a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:08.897208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05fba4-9330-442e-b7a2-b587921e8944 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:08.977829 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-3a3e2873-56ee-44e1-9555-fbc9cec16a53 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:08.981416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3a3e2873-56ee-44e1-9555-fbc9cec16a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:09.125678 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0948160-266c-4d8b-93f9-97787353e26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:09.126498 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0948160-266c-4d8b-93f9-97787353e26b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:09.126683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0948160-266c-4d8b-93f9-97787353e26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:09.126918 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0948160-266c-4d8b-93f9-97787353e26b admin admin] PATH_INFO: `/v3/projects/3fb1da0ac63a44fcb786d3cd4be30142` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:09.134623 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0948160-266c-4d8b-93f9-97787353e26b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fb1da0ac63a44fcb786d3cd4be30142)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:09.136719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0948160-266c-4d8b-93f9-97787353e26b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:09.157453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:09.176637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:09.204300 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 862/1724] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:09.371786 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e0948160-266c-4d8b-93f9-97787353e26b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fb1da0ac63a44fcb786d3cd4be30142', 'request_id': 'req-e0948160-266c-4d8b-93f9-97787353e26b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:09.422912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e0948160-266c-4d8b-93f9-97787353e26b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fb1da0ac63a44fcb786d3cd4be30142', 'request_id': 'req-e0948160-266c-4d8b-93f9-97787353e26b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:09.427612 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e2873-56ee-44e1-9555-fbc9cec16a53 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:09.434245 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3a3e2873-56ee-44e1-9555-fbc9cec16a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:09.451361 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e0948160-266c-4d8b-93f9-97787353e26b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fb1da0ac63a44fcb786d3cd4be30142', 'request_id': 'req-e0948160-266c-4d8b-93f9-97787353e26b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:09.487314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e0948160-266c-4d8b-93f9-97787353e26b admin admin] The token cache is being invalidate because project 3fb1da0ac63a44fcb786d3cd4be30142 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:09.488479 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e0948160-266c-4d8b-93f9-97787353e26b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:09.492339 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e0948160-266c-4d8b-93f9-97787353e26b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:09.497369 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 863/1725] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:11:08 2022] DELETE /identity/v3/projects/3fb1da0ac63a44fcb786d3cd4be30142 => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:09.498284 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0948160-266c-4d8b-93f9-97787353e26b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:09.509193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e0948160-266c-4d8b-93f9-97787353e26b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:09.622982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e2873-56ee-44e1-9555-fbc9cec16a53 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:09.672618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0948160-266c-4d8b-93f9-97787353e26b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:09.682502 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85323897-d140-46f8-aaa8-2e20aff3692e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:09.683066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85323897-d140-46f8-aaa8-2e20aff3692e service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:09.683512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85323897-d140-46f8-aaa8-2e20aff3692e service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:09.683806 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85323897-d140-46f8-aaa8-2e20aff3692e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:09.688139 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-85323897-d140-46f8-aaa8-2e20aff3692e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:09.740894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca88ba-eaf4-43a7-b3b6-23b7aa739a7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:09.741999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca88ba-eaf4-43a7-b3b6-23b7aa739a7a service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:09.742580 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca88ba-eaf4-43a7-b3b6-23b7aa739a7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:09.743665 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca88ba-eaf4-43a7-b3b6-23b7aa739a7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:09.746691 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-81ca88ba-eaf4-43a7-b3b6-23b7aa739a7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:09.860380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca88ba-eaf4-43a7-b3b6-23b7aa739a7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:09.863733 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca88ba-eaf4-43a7-b3b6-23b7aa739a7a service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:09.880990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 863/1726] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:09 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:09.882331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca88ba-eaf4-43a7-b3b6-23b7aa739a7a service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:09.886454 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-81ca88ba-eaf4-43a7-b3b6-23b7aa739a7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:09.893043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85323897-d140-46f8-aaa8-2e20aff3692e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:09.896586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85323897-d140-46f8-aaa8-2e20aff3692e service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:09.906384 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca88ba-eaf4-43a7-b3b6-23b7aa739a7a service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:09.913791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ecaf3-0ee4-4db7-8961-64a138ac4971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:09.928161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ecaf3-0ee4-4db7-8961-64a138ac4971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:09.928805 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ecaf3-0ee4-4db7-8961-64a138ac4971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:09.929650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ecaf3-0ee4-4db7-8961-64a138ac4971 admin admin] PATH_INFO: `/v3/users/5bcbbb365c124b1d9987c31315571409` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:09.931287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ecaf3-0ee4-4db7-8961-64a138ac4971 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bcbbb365c124b1d9987c31315571409)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:09.934798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ecaf3-0ee4-4db7-8961-64a138ac4971 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:09.937923 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 864/1727] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:09.938986 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85323897-d140-46f8-aaa8-2e20aff3692e service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:09.941853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-85323897-d140-46f8-aaa8-2e20aff3692e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:10.092655 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-c45ecaf3-0ee4-4db7-8961-64a138ac4971 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bcbbb365c124b1d9987c31315571409'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:10.095181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-c45ecaf3-0ee4-4db7-8961-64a138ac4971 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bcbbb365c124b1d9987c31315571409'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:10.101677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:10.120250 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-c45ecaf3-0ee4-4db7-8961-64a138ac4971 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bcbbb365c124b1d9987c31315571409'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:10.133644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:10.160415 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-c45ecaf3-0ee4-4db7-8961-64a138ac4971 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bcbbb365c124b1d9987c31315571409'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:10.163787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85323897-d140-46f8-aaa8-2e20aff3692e service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:10.169845 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53521115-5d05-407e-9a32-eda47534d312 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b40981fe179a43dc81553d3594dad649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:10.170631 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53521115-5d05-407e-9a32-eda47534d312 service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:10.170936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53521115-5d05-407e-9a32-eda47534d312 service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:10.171204 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53521115-5d05-407e-9a32-eda47534d312 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:10.178354 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-53521115-5d05-407e-9a32-eda47534d312 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:10.188058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-c45ecaf3-0ee4-4db7-8961-64a138ac4971 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bcbbb365c124b1d9987c31315571409'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:10.210470 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53521115-5d05-407e-9a32-eda47534d312 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:10.212057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53521115-5d05-407e-9a32-eda47534d312 service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:10.215862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 864/1728] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:11:09 2022] DELETE /identity/v3/users/5bcbbb365c124b1d9987c31315571409 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:10.237470 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 865/1729] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:09 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:10.431778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2921cb-b99b-41d9-bf01-43d1c622c044 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:10.432289 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2921cb-b99b-41d9-bf01-43d1c622c044 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:10.432904 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2921cb-b99b-41d9-bf01-43d1c622c044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:10.561306 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-ba2921cb-b99b-41d9-bf01-43d1c622c044 None None] MFA Rules not processed for user `0e93d159e8644203808918c5ae86fc74`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:10.567494 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ba2921cb-b99b-41d9-bf01-43d1c622c044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:10.777532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 865/1730] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:10 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:10.791488 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53521115-5d05-407e-9a32-eda47534d312 service placement] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:10.792982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-53521115-5d05-407e-9a32-eda47534d312 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:11.046553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2921cb-b99b-41d9-bf01-43d1c622c044 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:11.048050 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ba2921cb-b99b-41d9-bf01-43d1c622c044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:11.071723 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53521115-5d05-407e-9a32-eda47534d312 service placement] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:11.367113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92d6eb-4730-4ac2-a234-d1cf028fab51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:11.371068 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92d6eb-4730-4ac2-a234-d1cf028fab51 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:11.371360 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92d6eb-4730-4ac2-a234-d1cf028fab51 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:11.373246 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92d6eb-4730-4ac2-a234-d1cf028fab51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:11.374504 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2921cb-b99b-41d9-bf01-43d1c622c044 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:11.397754 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3370712a-738a-458a-8058-6ccb58dd1efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:11.398535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370712a-738a-458a-8058-6ccb58dd1efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:11.398919 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370712a-738a-458a-8058-6ccb58dd1efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:11.399299 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370712a-738a-458a-8058-6ccb58dd1efa admin admin] PATH_INFO: `/v3/projects/2d8d7b19037b4a1b95bfbf97fd624645` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:11.402125 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3370712a-738a-458a-8058-6ccb58dd1efa admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d8d7b19037b4a1b95bfbf97fd624645)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:11.414535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92d6eb-4730-4ac2-a234-d1cf028fab51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:11.421335 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92d6eb-4730-4ac2-a234-d1cf028fab51 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:11.421769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3370712a-738a-458a-8058-6ccb58dd1efa admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:11.486703 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 866/1731] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:10 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 695 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:11:11.571610 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92d6eb-4730-4ac2-a234-d1cf028fab51 service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:11.575624 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5e92d6eb-4730-4ac2-a234-d1cf028fab51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:11.651815 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92d6eb-4730-4ac2-a234-d1cf028fab51 service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:11.685725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3370712a-738a-458a-8058-6ccb58dd1efa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d8d7b19037b4a1b95bfbf97fd624645', 'request_id': 'req-3370712a-738a-458a-8058-6ccb58dd1efa'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:11.696250 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3370712a-738a-458a-8058-6ccb58dd1efa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d8d7b19037b4a1b95bfbf97fd624645', 'request_id': 'req-3370712a-738a-458a-8058-6ccb58dd1efa'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:11.712687 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3370712a-738a-458a-8058-6ccb58dd1efa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d8d7b19037b4a1b95bfbf97fd624645', 'request_id': 'req-3370712a-738a-458a-8058-6ccb58dd1efa'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:11.732965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:11.773392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3370712a-738a-458a-8058-6ccb58dd1efa admin admin] The token cache is being invalidate because project 2d8d7b19037b4a1b95bfbf97fd624645 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:11.773826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3370712a-738a-458a-8058-6ccb58dd1efa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:11.776460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3370712a-738a-458a-8058-6ccb58dd1efa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:11.783819 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 866/1732] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:11:11 2022] DELETE /identity/v3/projects/2d8d7b19037b4a1b95bfbf97fd624645 => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:11:11.789523 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3370712a-738a-458a-8058-6ccb58dd1efa admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:11.797332 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3370712a-738a-458a-8058-6ccb58dd1efa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:12.005028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340782e-b15e-47b2-9b33-c3b3a501e21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:12.018226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340782e-b15e-47b2-9b33-c3b3a501e21d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:12.032257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340782e-b15e-47b2-9b33-c3b3a501e21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:12.032757 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340782e-b15e-47b2-9b33-c3b3a501e21d admin admin] PATH_INFO: `/v3/projects/eae1791d3b4948afaf496d147ef2906b` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:12.038137 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f340782e-b15e-47b2-9b33-c3b3a501e21d admin admin] RBAC: Authorizing `identity:delete_project(project_id=eae1791d3b4948afaf496d147ef2906b)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:12.042389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f340782e-b15e-47b2-9b33-c3b3a501e21d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:12.062457 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:12.135845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3370712a-738a-458a-8058-6ccb58dd1efa admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:12.150608 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c25da-c4ea-41b7-ad10-b3891d955e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:12.153647 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c25da-c4ea-41b7-ad10-b3891d955e9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:12.156328 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c25da-c4ea-41b7-ad10-b3891d955e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:12.156328 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c25da-c4ea-41b7-ad10-b3891d955e9b admin admin] PATH_INFO: `/v3/users/1804fd3fa0854b90b17eb85b3dd1a59d` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:12.161556 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c25da-c4ea-41b7-ad10-b3891d955e9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1804fd3fa0854b90b17eb85b3dd1a59d)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:12.164206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c25da-c4ea-41b7-ad10-b3891d955e9b admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:12.252343 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f340782e-b15e-47b2-9b33-c3b3a501e21d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eae1791d3b4948afaf496d147ef2906b', 'request_id': 'req-f340782e-b15e-47b2-9b33-c3b3a501e21d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:12.274003 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f340782e-b15e-47b2-9b33-c3b3a501e21d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eae1791d3b4948afaf496d147ef2906b', 'request_id': 'req-f340782e-b15e-47b2-9b33-c3b3a501e21d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:12.309360 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f340782e-b15e-47b2-9b33-c3b3a501e21d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eae1791d3b4948afaf496d147ef2906b', 'request_id': 'req-f340782e-b15e-47b2-9b33-c3b3a501e21d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:12.365039 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f340782e-b15e-47b2-9b33-c3b3a501e21d admin admin] The token cache is being invalidate because project eae1791d3b4948afaf496d147ef2906b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:12.366890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f340782e-b15e-47b2-9b33-c3b3a501e21d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:12.370619 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f340782e-b15e-47b2-9b33-c3b3a501e21d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:12.371287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8a8c25da-c4ea-41b7-ad10-b3891d955e9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1804fd3fa0854b90b17eb85b3dd1a59d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:12.373561 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 867/1733] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:11:11 2022] DELETE /identity/v3/projects/eae1791d3b4948afaf496d147ef2906b => generated 0 bytes in 803 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:11:12.374185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8a8c25da-c4ea-41b7-ad10-b3891d955e9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1804fd3fa0854b90b17eb85b3dd1a59d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:12.383498 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9a776-e472-40f9-868a-9c1f8b27392b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:12.384963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9a776-e472-40f9-868a-9c1f8b27392b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:12.386342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9a776-e472-40f9-868a-9c1f8b27392b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:12.398511 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8a8c25da-c4ea-41b7-ad10-b3891d955e9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1804fd3fa0854b90b17eb85b3dd1a59d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:12.455142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8a8c25da-c4ea-41b7-ad10-b3891d955e9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1804fd3fa0854b90b17eb85b3dd1a59d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:12.465070 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-68d9a776-e472-40f9-868a-9c1f8b27392b None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:12.466468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-68d9a776-e472-40f9-868a-9c1f8b27392b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:12.480441 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8a8c25da-c4ea-41b7-ad10-b3891d955e9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1804fd3fa0854b90b17eb85b3dd1a59d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:12.518712 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 867/1734] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:11:11 2022] DELETE /identity/v3/users/1804fd3fa0854b90b17eb85b3dd1a59d => generated 0 bytes in 727 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:12.678094 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 868/1735] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:11:12.692572 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3080e-fa4a-4ade-ba88-9d637752f20c None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:12.692928 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3080e-fa4a-4ade-ba88-9d637752f20c None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:12.693241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3080e-fa4a-4ade-ba88-9d637752f20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:12.790705 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-85a3080e-fa4a-4ade-ba88-9d637752f20c None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:12.796390 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-85a3080e-fa4a-4ade-ba88-9d637752f20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:13.085468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9a776-e472-40f9-868a-9c1f8b27392b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:13.094154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 868/1736] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:13.136521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9a776-e472-40f9-868a-9c1f8b27392b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:13.170298 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22e1f3-b901-4ebd-983f-6d079974edb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:13.172670 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22e1f3-b901-4ebd-983f-6d079974edb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:13.174777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22e1f3-b901-4ebd-983f-6d079974edb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:13.175358 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22e1f3-b901-4ebd-983f-6d079974edb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:13.179901 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c22e1f3-b901-4ebd-983f-6d079974edb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:11:13.180463 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c22e1f3-b901-4ebd-983f-6d079974edb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:13.185948 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c22e1f3-b901-4ebd-983f-6d079974edb4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:13.241772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 869/1737] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:11:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:11:13.244047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3080e-fa4a-4ade-ba88-9d637752f20c None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:13.284050 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3080e-fa4a-4ade-ba88-9d637752f20c None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:13.299746 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454f624-799a-4c51-b127-e7563f2719fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:13.301280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454f624-799a-4c51-b127-e7563f2719fe admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:13.301707 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454f624-799a-4c51-b127-e7563f2719fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:13.302089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454f624-799a-4c51-b127-e7563f2719fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:13.302897 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454f624-799a-4c51-b127-e7563f2719fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:13.305794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454f624-799a-4c51-b127-e7563f2719fe admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:13.346952 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:13.374642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 869/1738] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:11:13 2022] POST /identity/v3/projects => generated 364 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:13.376042 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22e1f3-b901-4ebd-983f-6d079974edb4 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:13.410503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22e1f3-b901-4ebd-983f-6d079974edb4 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:13.596429 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ef421-2753-437e-9eb5-6172d880310c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:13.598127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ef421-2753-437e-9eb5-6172d880310c admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:13.598595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ef421-2753-437e-9eb5-6172d880310c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:13.599017 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ef421-2753-437e-9eb5-6172d880310c admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:13.599901 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ef421-2753-437e-9eb5-6172d880310c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:13.602063 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ef421-2753-437e-9eb5-6172d880310c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:13.626883 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aff8da-a3d8-4a79-b8fe-4e4e3192c64c None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:13.627182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aff8da-a3d8-4a79-b8fe-4e4e3192c64c None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:13.627461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aff8da-a3d8-4a79-b8fe-4e4e3192c64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:13.715710 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 870/1739] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:11:13 2022] POST /identity/v3/users => generated 283 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:13.716618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ef421-2753-437e-9eb5-6172d880310c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:13.731967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-94aff8da-a3d8-4a79-b8fe-4e4e3192c64c None None] MFA Rules not processed for user `6c30f2ab48e74d4690aafe46c4bf682f`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:13.733528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-94aff8da-a3d8-4a79-b8fe-4e4e3192c64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:13.748440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ef421-2753-437e-9eb5-6172d880310c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:13.759382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439956b9-c84b-4b0c-a6b0-9bb21a32c7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:13.761877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439956b9-c84b-4b0c-a6b0-9bb21a32c7ae admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:13.762372 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439956b9-c84b-4b0c-a6b0-9bb21a32c7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:13.763749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439956b9-c84b-4b0c-a6b0-9bb21a32c7ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:13.764654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439956b9-c84b-4b0c-a6b0-9bb21a32c7ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:13.769470 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439956b9-c84b-4b0c-a6b0-9bb21a32c7ae admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:13.795465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:13.819562 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 871/1740] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:13 2022] GET /identity/v3/roles => generated 1427 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:13.820548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439956b9-c84b-4b0c-a6b0-9bb21a32c7ae admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:13.851877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439956b9-c84b-4b0c-a6b0-9bb21a32c7ae admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:13.863246 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fc747-3db1-47eb-98f7-4627704b01bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:13.864828 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fc747-3db1-47eb-98f7-4627704b01bc admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:13.865316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fc747-3db1-47eb-98f7-4627704b01bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:13.865762 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fc747-3db1-47eb-98f7-4627704b01bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:13.866657 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680fc747-3db1-47eb-98f7-4627704b01bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:13.870460 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680fc747-3db1-47eb-98f7-4627704b01bc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:13.899051 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 872/1741] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:13 2022] GET /identity/v3/roles => generated 1427 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:13.907941 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fc747-3db1-47eb-98f7-4627704b01bc admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:13.943755 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fc747-3db1-47eb-98f7-4627704b01bc admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:13.944805 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 870/1742] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:13 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:13.949138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031b91d-c4d5-4a05-bb09-6f925f1d4327 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:13.949435 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031b91d-c4d5-4a05-bb09-6f925f1d4327 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:13.949721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031b91d-c4d5-4a05-bb09-6f925f1d4327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:13.968433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cc012-a4bb-42ff-b47d-eacd0bf7feac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:13.969414 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cc012-a4bb-42ff-b47d-eacd0bf7feac admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:13.969696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cc012-a4bb-42ff-b47d-eacd0bf7feac admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:13.970142 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cc012-a4bb-42ff-b47d-eacd0bf7feac admin admin] PATH_INFO: `/v3/projects/19f61c15161a40719159ed7579d62fa9/users/5e589f8ba5cf4d3b99c3272e35e8d50d/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:14.012006 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-3031b91d-c4d5-4a05-bb09-6f925f1d4327 None None] MFA Rules not processed for user `6c30f2ab48e74d4690aafe46c4bf682f`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:14.015018 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3031b91d-c4d5-4a05-bb09-6f925f1d4327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:14.018045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8cc012-a4bb-42ff-b47d-eacd0bf7feac admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f61c15161a40719159ed7579d62fa9, user_id=5e589f8ba5cf4d3b99c3272e35e8d50d, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:14.022875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8cc012-a4bb-42ff-b47d-eacd0bf7feac admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:14.053405 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 871/1743] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:13 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:14.056299 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031b91d-c4d5-4a05-bb09-6f925f1d4327 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:14.059482 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3031b91d-c4d5-4a05-bb09-6f925f1d4327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:14.097528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 873/1744] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:13 2022] PUT /identity/v3/projects/19f61c15161a40719159ed7579d62fa9/users/5e589f8ba5cf4d3b99c3272e35e8d50d/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:14.097528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cc012-a4bb-42ff-b47d-eacd0bf7feac admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:14.109137 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ae8cc012-a4bb-42ff-b47d-eacd0bf7feac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:14.110976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031b91d-c4d5-4a05-bb09-6f925f1d4327 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:14.262332 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cc012-a4bb-42ff-b47d-eacd0bf7feac admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:14.281586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8d103-f423-4407-be47-e40ec801976e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:14.282257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8d103-f423-4407-be47-e40ec801976e service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:14.282729 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8d103-f423-4407-be47-e40ec801976e service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:14.283275 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8d103-f423-4407-be47-e40ec801976e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:14.292614 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1bf7bc-fbf8-4c4c-a328-416a1b182402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:14.293547 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bf7bc-fbf8-4c4c-a328-416a1b182402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:14.293950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bf7bc-fbf8-4c4c-a328-416a1b182402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:14.294960 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bf7bc-fbf8-4c4c-a328-416a1b182402 admin admin] PATH_INFO: `/v3/projects/04e129d8b34941fba640d81a283f0773` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:14.296904 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1bf7bc-fbf8-4c4c-a328-416a1b182402 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04e129d8b34941fba640d81a283f0773)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:14.310443 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1bf7bc-fbf8-4c4c-a328-416a1b182402 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:14.423634 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8d103-f423-4407-be47-e40ec801976e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:14.426102 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8d103-f423-4407-be47-e40ec801976e service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:14.437808 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-da1bf7bc-fbf8-4c4c-a328-416a1b182402 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04e129d8b34941fba640d81a283f0773', 'request_id': 'req-da1bf7bc-fbf8-4c4c-a328-416a1b182402'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:14.448657 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 874/1745] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:14 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:14.463811 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457241d-5696-4c87-8582-174bd7c6ed49 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:14.464406 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457241d-5696-4c87-8582-174bd7c6ed49 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:14.465195 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-da1bf7bc-fbf8-4c4c-a328-416a1b182402 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04e129d8b34941fba640d81a283f0773', 'request_id': 'req-da1bf7bc-fbf8-4c4c-a328-416a1b182402'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:14.465612 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457241d-5696-4c87-8582-174bd7c6ed49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:14.489054 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-da1bf7bc-fbf8-4c4c-a328-416a1b182402 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04e129d8b34941fba640d81a283f0773', 'request_id': 'req-da1bf7bc-fbf8-4c4c-a328-416a1b182402'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:14.529738 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-da1bf7bc-fbf8-4c4c-a328-416a1b182402 admin admin] The token cache is being invalidate because project 04e129d8b34941fba640d81a283f0773 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:14.530194 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-da1bf7bc-fbf8-4c4c-a328-416a1b182402 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:14.533617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-da1bf7bc-fbf8-4c4c-a328-416a1b182402 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:14.552974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 872/1746] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:11:14 2022] DELETE /identity/v3/projects/04e129d8b34941fba640d81a283f0773 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:14.554727 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e219f-71f1-4708-b0b5-93ffcd073a3c None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:14.569027 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e219f-71f1-4708-b0b5-93ffcd073a3c None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:14.570639 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e219f-71f1-4708-b0b5-93ffcd073a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:14.607113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5457241d-5696-4c87-8582-174bd7c6ed49 None None] MFA Rules not processed for user `5e589f8ba5cf4d3b99c3272e35e8d50d`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:14.608705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5457241d-5696-4c87-8582-174bd7c6ed49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:14.635682 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:14.642805 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-1c6e219f-71f1-4708-b0b5-93ffcd073a3c None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:14.645492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1c6e219f-71f1-4708-b0b5-93ffcd073a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:14.772549 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 873/1747] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:14.782565 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:14.786193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff281dc-6ce3-40b3-a749-74476abab2e1 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:14.786811 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff281dc-6ce3-40b3-a749-74476abab2e1 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:14.787162 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff281dc-6ce3-40b3-a749-74476abab2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:14.812731 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 875/1748] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:14.817827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457241d-5696-4c87-8582-174bd7c6ed49 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:14.819687 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5457241d-5696-4c87-8582-174bd7c6ed49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:14.821487 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-7ff281dc-6ce3-40b3-a749-74476abab2e1 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:14.822905 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-7ff281dc-6ce3-40b3-a749-74476abab2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:14.836357 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 874/1749] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:14.840420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff281dc-6ce3-40b3-a749-74476abab2e1 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:14.853417 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457241d-5696-4c87-8582-174bd7c6ed49 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:14.864076 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b3942-3058-4c71-b5a5-df531f2723b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:14.864878 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff281dc-6ce3-40b3-a749-74476abab2e1 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:14.864990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b3942-3058-4c71-b5a5-df531f2723b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:14.864990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b3942-3058-4c71-b5a5-df531f2723b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:14.865307 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b3942-3058-4c71-b5a5-df531f2723b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:14.869083 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b3942-3058-4c71-b5a5-df531f2723b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:14.871123 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b3942-3058-4c71-b5a5-df531f2723b8 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:14.872336 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe0da3-a3c9-46c7-88ec-3fcb04e17749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:14.873074 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe0da3-a3c9-46c7-88ec-3fcb04e17749 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:14.873457 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe0da3-a3c9-46c7-88ec-3fcb04e17749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:14.873809 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe0da3-a3c9-46c7-88ec-3fcb04e17749 admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:14.874600 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe0da3-a3c9-46c7-88ec-3fcb04e17749 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:11:14.875113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe0da3-a3c9-46c7-88ec-3fcb04e17749 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:14.876962 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe0da3-a3c9-46c7-88ec-3fcb04e17749 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:14.895132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 875/1750] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:11:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:11:14.896578 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe0da3-a3c9-46c7-88ec-3fcb04e17749 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:14.929998 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe0da3-a3c9-46c7-88ec-3fcb04e17749 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:14.942220 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 876/1751] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:11:14 2022] POST /identity/v3/projects => generated 362 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:11:14.943871 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b3942-3058-4c71-b5a5-df531f2723b8 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:14.944939 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82faa4e-2fa7-4c27-bcba-6b0740156214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:14.945603 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82faa4e-2fa7-4c27-bcba-6b0740156214 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:14.945879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82faa4e-2fa7-4c27-bcba-6b0740156214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:14.946155 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82faa4e-2fa7-4c27-bcba-6b0740156214 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:14.947143 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82faa4e-2fa7-4c27-bcba-6b0740156214 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:14.949204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82faa4e-2fa7-4c27-bcba-6b0740156214 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:15.000286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 876/1752] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:11:14 2022] POST /identity/v3/projects => generated 384 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:15.003729 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a4a49-ad00-4f31-97e1-5f7b01a6a142 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:15.004611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a4a49-ad00-4f31-97e1-5f7b01a6a142 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:15.005995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a4a49-ad00-4f31-97e1-5f7b01a6a142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:15.046275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-fa3a4a49-ad00-4f31-97e1-5f7b01a6a142 None None] MFA Rules not processed for user `c9633b9cbd8a4332a605ca66d73dc5b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:15.049179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-fa3a4a49-ad00-4f31-97e1-5f7b01a6a142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:15.156753 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b3942-3058-4c71-b5a5-df531f2723b8 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:15.158923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 877/1753] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:15.160648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a4a49-ad00-4f31-97e1-5f7b01a6a142 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:15.260470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:15.267316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a4a49-ad00-4f31-97e1-5f7b01a6a142 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:15.274394 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af754329-42f6-4dc7-bf20-2daf71366ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:15.275230 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af754329-42f6-4dc7-bf20-2daf71366ba1 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:15.275763 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af754329-42f6-4dc7-bf20-2daf71366ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:15.276118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af754329-42f6-4dc7-bf20-2daf71366ba1 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:15.276943 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af754329-42f6-4dc7-bf20-2daf71366ba1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:15.279287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af754329-42f6-4dc7-bf20-2daf71366ba1 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:15.309173 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 878/1754] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:11:15 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:15.310770 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af754329-42f6-4dc7-bf20-2daf71366ba1 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:15.312517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-af754329-42f6-4dc7-bf20-2daf71366ba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:15.316348 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9563225-ffdc-4db8-a231-56ece98c0ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:15.318430 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9563225-ffdc-4db8-a231-56ece98c0ed6 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:15.319031 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9563225-ffdc-4db8-a231-56ece98c0ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:15.319594 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9563225-ffdc-4db8-a231-56ece98c0ed6 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:15.321047 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9563225-ffdc-4db8-a231-56ece98c0ed6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:15.330698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9563225-ffdc-4db8-a231-56ece98c0ed6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:15.429583 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 877/1755] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:11:14 2022] POST /identity/v3/users => generated 288 bytes in 487 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:15.433880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9563225-ffdc-4db8-a231-56ece98c0ed6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:15.529160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9563225-ffdc-4db8-a231-56ece98c0ed6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:15.569444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe862be9-2cf9-4bf9-a55b-e73c16a967f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:15.578622 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe862be9-2cf9-4bf9-a55b-e73c16a967f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:15.579075 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe862be9-2cf9-4bf9-a55b-e73c16a967f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:15.579517 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe862be9-2cf9-4bf9-a55b-e73c16a967f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:15.589847 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe862be9-2cf9-4bf9-a55b-e73c16a967f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:15.604096 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe862be9-2cf9-4bf9-a55b-e73c16a967f6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:15.646559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 878/1756] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:15 2022] GET /identity/v3/roles => generated 1427 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:15.647826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe862be9-2cf9-4bf9-a55b-e73c16a967f6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:15.679082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af754329-42f6-4dc7-bf20-2daf71366ba1 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:15.679284 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe862be9-2cf9-4bf9-a55b-e73c16a967f6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:15.686293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd94c2-f3eb-47b3-b334-294ab01b9d87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:15.687421 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd94c2-f3eb-47b3-b334-294ab01b9d87 service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:15.687880 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd94c2-f3eb-47b3-b334-294ab01b9d87 service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:15.688305 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd94c2-f3eb-47b3-b334-294ab01b9d87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:15.698904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658073d2-437c-450a-acb1-8c80e02831f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:15.699705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658073d2-437c-450a-acb1-8c80e02831f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:15.699830 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658073d2-437c-450a-acb1-8c80e02831f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:15.702154 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658073d2-437c-450a-acb1-8c80e02831f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:15.706207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658073d2-437c-450a-acb1-8c80e02831f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:15.710000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658073d2-437c-450a-acb1-8c80e02831f5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:15.712711 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd94c2-f3eb-47b3-b334-294ab01b9d87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:15.716649 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd94c2-f3eb-47b3-b334-294ab01b9d87 service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:15.761625 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 879/1757] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:15 2022] GET /identity/v3/roles => generated 1427 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:15.762657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 879/1758] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:15 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:15.764084 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658073d2-437c-450a-acb1-8c80e02831f5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:15.764546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd94c2-f3eb-47b3-b334-294ab01b9d87 service nova] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:15.803277 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658073d2-437c-450a-acb1-8c80e02831f5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:15.804494 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd94c2-f3eb-47b3-b334-294ab01b9d87 service nova] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:15.820171 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7fded-35a8-44b0-b0d8-b6736d5dc35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:15.821933 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b9d18-dcac-4fa3-829c-1e75417518de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:15.823739 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7fded-35a8-44b0-b0d8-b6736d5dc35b admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:15.824064 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7fded-35a8-44b0-b0d8-b6736d5dc35b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:15.824414 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7fded-35a8-44b0-b0d8-b6736d5dc35b admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:15.826308 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab7fded-35a8-44b0-b0d8-b6736d5dc35b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:15.830480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab7fded-35a8-44b0-b0d8-b6736d5dc35b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:15.834798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b9d18-dcac-4fa3-829c-1e75417518de admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:15.835133 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b9d18-dcac-4fa3-829c-1e75417518de admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:15.835383 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b9d18-dcac-4fa3-829c-1e75417518de admin admin] PATH_INFO: `/v3/projects/cafcb6d6ac6f4d39b0b8ebc2ad891934/users/cf1f6180d0ca49b6b5afc5249b8cee21/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:15.879570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b9d18-dcac-4fa3-829c-1e75417518de admin admin] RBAC: Authorizing `identity:create_grant(project_id=cafcb6d6ac6f4d39b0b8ebc2ad891934, user_id=cf1f6180d0ca49b6b5afc5249b8cee21, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:15.879736 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 880/1759] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:15 2022] GET /identity/v3/roles => generated 1427 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:15.882587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7fded-35a8-44b0-b0d8-b6736d5dc35b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:15.888721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b9d18-dcac-4fa3-829c-1e75417518de admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:15.958533 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 880/1760] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:15 2022] PUT /identity/v3/projects/cafcb6d6ac6f4d39b0b8ebc2ad891934/users/cf1f6180d0ca49b6b5afc5249b8cee21/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:15.962523 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b9d18-dcac-4fa3-829c-1e75417518de admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:15.972510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7fded-35a8-44b0-b0d8-b6736d5dc35b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:16.138661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:16.163868 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b9d18-dcac-4fa3-829c-1e75417518de admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:16.172267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d50d9-df6f-4154-9cff-8075e44db5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:16.173689 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d50d9-df6f-4154-9cff-8075e44db5c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:16.174175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d50d9-df6f-4154-9cff-8075e44db5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:16.174782 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d50d9-df6f-4154-9cff-8075e44db5c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:16.176163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188d50d9-df6f-4154-9cff-8075e44db5c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:16.180850 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188d50d9-df6f-4154-9cff-8075e44db5c5 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:16.200523 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 881/1761] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:15 2022] GET /identity/v3/roles => generated 1427 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:11:16.207024 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d50d9-df6f-4154-9cff-8075e44db5c5 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:16.238365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff3c00-7f19-4ec5-81ae-3ff126786493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:16.239021 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff3c00-7f19-4ec5-81ae-3ff126786493 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:16.239310 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff3c00-7f19-4ec5-81ae-3ff126786493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:16.239607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff3c00-7f19-4ec5-81ae-3ff126786493 admin admin] PATH_INFO: `/v3/projects/9b89fc0977074abda8fb75608df14d2b/users/edfa0775f5694c08b78ff30ca448cdf6/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:16.270109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d50d9-df6f-4154-9cff-8075e44db5c5 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:16.283809 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d0b89-40f8-446f-a75a-58c49dda15d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:16.284575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d0b89-40f8-446f-a75a-58c49dda15d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:16.285500 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d0b89-40f8-446f-a75a-58c49dda15d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:16.285500 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d0b89-40f8-446f-a75a-58c49dda15d0 admin admin] PATH_INFO: `/v3/projects/cafcb6d6ac6f4d39b0b8ebc2ad891934/users/cf1f6180d0ca49b6b5afc5249b8cee21/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:16.287730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff3c00-7f19-4ec5-81ae-3ff126786493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b89fc0977074abda8fb75608df14d2b, user_id=edfa0775f5694c08b78ff30ca448cdf6, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:16.290035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d0b89-40f8-446f-a75a-58c49dda15d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cafcb6d6ac6f4d39b0b8ebc2ad891934, user_id=cf1f6180d0ca49b6b5afc5249b8cee21, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:16.290882 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff3c00-7f19-4ec5-81ae-3ff126786493 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:16.296297 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d0b89-40f8-446f-a75a-58c49dda15d0 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:16.318117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 881/1762] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:15 2022] PUT /identity/v3/projects/9b89fc0977074abda8fb75608df14d2b/users/edfa0775f5694c08b78ff30ca448cdf6/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:16.328603 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e382f8-5a78-4e54-8764-7c6410cabf30 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:16.328908 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e382f8-5a78-4e54-8764-7c6410cabf30 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:16.329292 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e382f8-5a78-4e54-8764-7c6410cabf30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:16.339202 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 882/1763] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:16 2022] PUT /identity/v3/projects/cafcb6d6ac6f4d39b0b8ebc2ad891934/users/cf1f6180d0ca49b6b5afc5249b8cee21/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:16.360390 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819bece-8768-4c67-a6cb-9ff0a602ce9a None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:16.360848 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819bece-8768-4c67-a6cb-9ff0a602ce9a None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:16.361306 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819bece-8768-4c67-a6cb-9ff0a602ce9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:16.453725 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f1e382f8-5a78-4e54-8764-7c6410cabf30 None None] MFA Rules not processed for user `edfa0775f5694c08b78ff30ca448cdf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:16.455865 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f1e382f8-5a78-4e54-8764-7c6410cabf30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:16.481495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-d819bece-8768-4c67-a6cb-9ff0a602ce9a None None] MFA Rules not processed for user `cf1f6180d0ca49b6b5afc5249b8cee21`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:16.482758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d819bece-8768-4c67-a6cb-9ff0a602ce9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:16.540092 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:16.738175 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 882/1764] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:16.742458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c57d1a-9364-4d46-8d06-8590170d7985 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:16.744630 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c57d1a-9364-4d46-8d06-8590170d7985 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:16.745306 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c57d1a-9364-4d46-8d06-8590170d7985 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:16.751727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 883/1765] 10.222.0.100 () {54 vars in 917 bytes} [Tue Aug 2 09:11:16 2022] GET /identity => generated 271 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:11:16.754795 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c57d1a-9364-4d46-8d06-8590170d7985 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:16.804611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 883/1766] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:16.809979 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eee3a3-e5c5-446c-8fba-41e6f04b8edc None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:16.809979 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eee3a3-e5c5-446c-8fba-41e6f04b8edc None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:16.810243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eee3a3-e5c5-446c-8fba-41e6f04b8edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:16.873867 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-53eee3a3-e5c5-446c-8fba-41e6f04b8edc None None] MFA Rules not processed for user `f4afef33deb247509765a8c88a78ac39`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:16.879249 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-53eee3a3-e5c5-446c-8fba-41e6f04b8edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:16.965680 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c57d1a-9364-4d46-8d06-8590170d7985 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:16.980963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0c3aa-a965-4b23-866b-d5a1629dcc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:16.984597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0c3aa-a965-4b23-866b-d5a1629dcc31 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:16.985125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0c3aa-a965-4b23-866b-d5a1629dcc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:16.985567 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0c3aa-a965-4b23-866b-d5a1629dcc31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:16.990062 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0c3aa-a965-4b23-866b-d5a1629dcc31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:16.993158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0c3aa-a965-4b23-866b-d5a1629dcc31 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:17.075786 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 884/1767] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:11:16 2022] POST /identity/v3/projects => generated 382 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:17.079205 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0c3aa-a965-4b23-866b-d5a1629dcc31 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:17.079967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 884/1768] 10.222.0.100 () {58 vars in 1014 bytes} [Tue Aug 2 09:11:16 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:17.084078 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eee3a3-e5c5-446c-8fba-41e6f04b8edc None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:17.084312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f9e0c3aa-a965-4b23-866b-d5a1629dcc31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:17.156094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eee3a3-e5c5-446c-8fba-41e6f04b8edc None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:17.238346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:17.310481 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0c3aa-a965-4b23-866b-d5a1629dcc31 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:17.320968 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea029b6-8333-4b33-a8d4-21f2967dcc74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:17.324610 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea029b6-8333-4b33-a8d4-21f2967dcc74 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:17.325402 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea029b6-8333-4b33-a8d4-21f2967dcc74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:17.327901 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea029b6-8333-4b33-a8d4-21f2967dcc74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:17.437539 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a68ee-75de-4a99-8b12-d89995d7a030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:17.438233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a68ee-75de-4a99-8b12-d89995d7a030 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:17.438520 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a68ee-75de-4a99-8b12-d89995d7a030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:17.438793 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a68ee-75de-4a99-8b12-d89995d7a030 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:17.443225 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a68ee-75de-4a99-8b12-d89995d7a030 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:17.449444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a68ee-75de-4a99-8b12-d89995d7a030 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:17.510493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 885/1769] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:11:17 2022] POST /identity/v3/users => generated 298 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:17.536058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a68ee-75de-4a99-8b12-d89995d7a030 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:17.546378 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d24a68ee-75de-4a99-8b12-d89995d7a030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:17.564279 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea029b6-8333-4b33-a8d4-21f2967dcc74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:17.565986 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea029b6-8333-4b33-a8d4-21f2967dcc74 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:17.577922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 885/1770] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:17.578834 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea029b6-8333-4b33-a8d4-21f2967dcc74 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:17.612162 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea029b6-8333-4b33-a8d4-21f2967dcc74 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:17.626416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1813237-cf2a-4e58-8a72-acf183f27c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:17.627184 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1813237-cf2a-4e58-8a72-acf183f27c96 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:17.627581 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1813237-cf2a-4e58-8a72-acf183f27c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:17.627870 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1813237-cf2a-4e58-8a72-acf183f27c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:17.628888 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1813237-cf2a-4e58-8a72-acf183f27c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:17.631190 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1813237-cf2a-4e58-8a72-acf183f27c96 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:17.659129 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 886/1771] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:17 2022] GET /identity/v3/roles => generated 1427 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:17.663165 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1813237-cf2a-4e58-8a72-acf183f27c96 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:17.689100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1813237-cf2a-4e58-8a72-acf183f27c96 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:17.695445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb46a4-6a21-48f4-a9b9-72ff5ee90a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:17.699219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb46a4-6a21-48f4-a9b9-72ff5ee90a9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:17.699750 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb46a4-6a21-48f4-a9b9-72ff5ee90a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:17.700199 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb46a4-6a21-48f4-a9b9-72ff5ee90a9e admin admin] PATH_INFO: `/v3/projects/4382a8dd6a5e4c89b54055f1708be577/users/b0d47787b92945b7a9474f8e1470e3e3/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:17.756321 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb46a4-6a21-48f4-a9b9-72ff5ee90a9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4382a8dd6a5e4c89b54055f1708be577, user_id=b0d47787b92945b7a9474f8e1470e3e3, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:17.768100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb46a4-6a21-48f4-a9b9-72ff5ee90a9e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:17.794492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 887/1772] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:17 2022] PUT /identity/v3/projects/4382a8dd6a5e4c89b54055f1708be577/users/b0d47787b92945b7a9474f8e1470e3e3/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:17.795965 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05940903-279c-4b0c-9a69-12181682acbb None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:17.796256 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05940903-279c-4b0c-9a69-12181682acbb None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:17.796538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05940903-279c-4b0c-9a69-12181682acbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:17.886413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a68ee-75de-4a99-8b12-d89995d7a030 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:17.898914 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:17.934293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-05940903-279c-4b0c-9a69-12181682acbb None None] MFA Rules not processed for user `1cd36b543bd9487ebd8ce578e9a76c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:17.935946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-05940903-279c-4b0c-9a69-12181682acbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:18.052805 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d8ac6-2213-421e-b5fe-77b1b9d59a22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:18.053495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d8ac6-2213-421e-b5fe-77b1b9d59a22 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:18.053776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d8ac6-2213-421e-b5fe-77b1b9d59a22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:18.054059 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d8ac6-2213-421e-b5fe-77b1b9d59a22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:18.073828 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d8ac6-2213-421e-b5fe-77b1b9d59a22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:18.079601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d8ac6-2213-421e-b5fe-77b1b9d59a22 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:18.082583 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 888/1773] 10.222.0.100 () {58 vars in 1014 bytes} [Tue Aug 2 09:11:17 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:18.088801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05940903-279c-4b0c-9a69-12181682acbb None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:18.110668 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 886/1774] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:17 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:18.116189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d8ac6-2213-421e-b5fe-77b1b9d59a22 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:18.133188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:18.304979 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05940903-279c-4b0c-9a69-12181682acbb None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:18.330919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7725b32-5d87-4244-b409-5909abb3be77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:18.333091 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7725b32-5d87-4244-b409-5909abb3be77 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:18.336178 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7725b32-5d87-4244-b409-5909abb3be77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:18.336583 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7725b32-5d87-4244-b409-5909abb3be77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:18.339437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d8ac6-2213-421e-b5fe-77b1b9d59a22 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:18.340273 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7725b32-5d87-4244-b409-5909abb3be77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:18.346005 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7725b32-5d87-4244-b409-5909abb3be77 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:18.356234 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf911afe-11b3-4109-9b3c-e6f5fba73f34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:18.356995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf911afe-11b3-4109-9b3c-e6f5fba73f34 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:18.360054 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf911afe-11b3-4109-9b3c-e6f5fba73f34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:18.360476 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf911afe-11b3-4109-9b3c-e6f5fba73f34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:18.381223 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 889/1775] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:18 2022] GET /identity/v3/roles => generated 1427 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:18.385627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7725b32-5d87-4244-b409-5909abb3be77 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:18.427609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7725b32-5d87-4244-b409-5909abb3be77 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:18.450969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50733ad-d9f9-42ee-9ece-6cc43bdf7169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:18.451680 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50733ad-d9f9-42ee-9ece-6cc43bdf7169 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:18.451970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50733ad-d9f9-42ee-9ece-6cc43bdf7169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:18.452258 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50733ad-d9f9-42ee-9ece-6cc43bdf7169 admin admin] PATH_INFO: `/v3/projects/4382a8dd6a5e4c89b54055f1708be577/users/b0d47787b92945b7a9474f8e1470e3e3/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:18.461050 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50733ad-d9f9-42ee-9ece-6cc43bdf7169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4382a8dd6a5e4c89b54055f1708be577, user_id=b0d47787b92945b7a9474f8e1470e3e3, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:18.471424 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50733ad-d9f9-42ee-9ece-6cc43bdf7169 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:18.516683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 890/1776] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:18 2022] PUT /identity/v3/projects/4382a8dd6a5e4c89b54055f1708be577/users/b0d47787b92945b7a9474f8e1470e3e3/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:18.520922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50733ad-d9f9-42ee-9ece-6cc43bdf7169 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:18.531018 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b50733ad-d9f9-42ee-9ece-6cc43bdf7169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:18.577185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf911afe-11b3-4109-9b3c-e6f5fba73f34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:18.581055 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf911afe-11b3-4109-9b3c-e6f5fba73f34 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:18.614866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 887/1777] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:18.615987 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a16c90-08db-4ad2-b0f9-e07900f06baf None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:18.616283 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a16c90-08db-4ad2-b0f9-e07900f06baf None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:18.616572 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a16c90-08db-4ad2-b0f9-e07900f06baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:18.680477 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-38a16c90-08db-4ad2-b0f9-e07900f06baf None None] MFA Rules not processed for user `0e93d159e8644203808918c5ae86fc74`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:18.681792 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-38a16c90-08db-4ad2-b0f9-e07900f06baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:18.820211 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50733ad-d9f9-42ee-9ece-6cc43bdf7169 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:18.839702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417aa789-d04f-44f3-8c8f-63c792071414 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50cb532b58ed4951ba960a0a7d579b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:18.840486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417aa789-d04f-44f3-8c8f-63c792071414 service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:18.840669 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417aa789-d04f-44f3-8c8f-63c792071414 service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:18.842410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417aa789-d04f-44f3-8c8f-63c792071414 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:18.851473 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 888/1778] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:18.855388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f461474-9c65-4c18-97c4-d4aacc742d34 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:18.855388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f461474-9c65-4c18-97c4-d4aacc742d34 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:18.855706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f461474-9c65-4c18-97c4-d4aacc742d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:18.962231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:18.992932 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-0f461474-9c65-4c18-97c4-d4aacc742d34 None None] MFA Rules not processed for user `b0d47787b92945b7a9474f8e1470e3e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:18.996484 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-0f461474-9c65-4c18-97c4-d4aacc742d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:19.097776 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417aa789-d04f-44f3-8c8f-63c792071414 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:19.099336 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417aa789-d04f-44f3-8c8f-63c792071414 service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:19.127888 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 891/1779] 10.222.0.100 () {60 vars in 1389 bytes} [Tue Aug 2 09:11:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:11:19.235273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 889/1780] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:18 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:19.248334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417aa789-d04f-44f3-8c8f-63c792071414 service swift] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:19.250489 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-417aa789-d04f-44f3-8c8f-63c792071414 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:19.350945 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:19.471236 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417aa789-d04f-44f3-8c8f-63c792071414 service swift] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:19.480322 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed68c00-38f0-48e7-bfeb-11c16adfa450 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:19.491553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed68c00-38f0-48e7-bfeb-11c16adfa450 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:19.491965 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed68c00-38f0-48e7-bfeb-11c16adfa450 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:19.494789 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed68c00-38f0-48e7-bfeb-11c16adfa450 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:19.514234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed68c00-38f0-48e7-bfeb-11c16adfa450 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:19.516801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed68c00-38f0-48e7-bfeb-11c16adfa450 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:19.543457 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 892/1781] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:19 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:19.833020 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f461474-9c65-4c18-97c4-d4aacc742d34 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:19.861756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f461474-9c65-4c18-97c4-d4aacc742d34 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:19.885036 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fbd99-2111-4b17-aeed-326cbcda1ddf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:19.885723 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fbd99-2111-4b17-aeed-326cbcda1ddf service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:19.886012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fbd99-2111-4b17-aeed-326cbcda1ddf service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:19.886295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fbd99-2111-4b17-aeed-326cbcda1ddf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:20.262227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fbd99-2111-4b17-aeed-326cbcda1ddf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:20.268155 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fbd99-2111-4b17-aeed-326cbcda1ddf service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:20.336804 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 890/1782] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:19 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:20.744585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c2559-dcbe-4701-9750-617df8bf381e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:20.745696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c2559-dcbe-4701-9750-617df8bf381e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:20.746138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c2559-dcbe-4701-9750-617df8bf381e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:20.814203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-718c2559-dcbe-4701-9750-617df8bf381e None None] MFA Rules not processed for user `f4afef33deb247509765a8c88a78ac39`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:20.828903 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-718c2559-dcbe-4701-9750-617df8bf381e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:21.048006 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 893/1783] 10.222.0.70 () {58 vars in 1019 bytes} [Tue Aug 2 09:11:20 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:21.091691 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fbd99-2111-4b17-aeed-326cbcda1ddf service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:21.097971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-7c5fbd99-2111-4b17-aeed-326cbcda1ddf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:21.151426 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fbd99-2111-4b17-aeed-326cbcda1ddf service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:21.175901 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03c028-6dd8-4d85-b903-85b244078b40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:21.176552 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03c028-6dd8-4d85-b903-85b244078b40 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:21.176844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03c028-6dd8-4d85-b903-85b244078b40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:21.177282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03c028-6dd8-4d85-b903-85b244078b40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:21.213714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03c028-6dd8-4d85-b903-85b244078b40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:21.220063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03c028-6dd8-4d85-b903-85b244078b40 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:21.268064 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 891/1784] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:21 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:21.304464 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136b679-9894-4302-bb22-e5baded336ac None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:21.305498 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136b679-9894-4302-bb22-e5baded336ac None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:21.306255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136b679-9894-4302-bb22-e5baded336ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:21.374661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e136b679-9894-4302-bb22-e5baded336ac None None] MFA Rules not processed for user `6c30f2ab48e74d4690aafe46c4bf682f`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:21.377132 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e136b679-9894-4302-bb22-e5baded336ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:21.521580 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 894/1785] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:21 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:21.755548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03c028-6dd8-4d85-b903-85b244078b40 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:22.040945 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03c028-6dd8-4d85-b903-85b244078b40 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:22.064366 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a90d60-06fb-49b1-ac5c-c4e65f8e3ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:22.065038 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a90d60-06fb-49b1-ac5c-c4e65f8e3ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:22.065345 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a90d60-06fb-49b1-ac5c-c4e65f8e3ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:22.066646 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a90d60-06fb-49b1-ac5c-c4e65f8e3ff6 admin admin] PATH_INFO: `/v3/users/edfa0775f5694c08b78ff30ca448cdf6` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:22.078070 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a90d60-06fb-49b1-ac5c-c4e65f8e3ff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edfa0775f5694c08b78ff30ca448cdf6)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:22.083599 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a90d60-06fb-49b1-ac5c-c4e65f8e3ff6 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:22.202225 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:22.317690 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-38a90d60-06fb-49b1-ac5c-c4e65f8e3ff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edfa0775f5694c08b78ff30ca448cdf6'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:22.320261 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-38a90d60-06fb-49b1-ac5c-c4e65f8e3ff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edfa0775f5694c08b78ff30ca448cdf6'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:22.352121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-38a90d60-06fb-49b1-ac5c-c4e65f8e3ff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edfa0775f5694c08b78ff30ca448cdf6'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:22.419272 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-38a90d60-06fb-49b1-ac5c-c4e65f8e3ff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edfa0775f5694c08b78ff30ca448cdf6'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:22.461317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-38a90d60-06fb-49b1-ac5c-c4e65f8e3ff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edfa0775f5694c08b78ff30ca448cdf6'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:22.494516 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 892/1786] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:11:21 2022] DELETE /identity/v3/users/edfa0775f5694c08b78ff30ca448cdf6 => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:22.500307 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc531e8-b188-4ec1-bbf3-0788deb3d62f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:22.500888 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc531e8-b188-4ec1-bbf3-0788deb3d62f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:22.501030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc531e8-b188-4ec1-bbf3-0788deb3d62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:22.666996 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8dc531e8-b188-4ec1-bbf3-0788deb3d62f None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:22.668015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8dc531e8-b188-4ec1-bbf3-0788deb3d62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:23.072705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 895/1787] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 574 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:23.167582 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a90d60-06fb-49b1-ac5c-c4e65f8e3ff6 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:23.169364 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-38a90d60-06fb-49b1-ac5c-c4e65f8e3ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:23.445473 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a90d60-06fb-49b1-ac5c-c4e65f8e3ff6 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:23.469732 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac935a4f-a2f7-485d-a5e5-b3fe75c265e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:23.476079 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac935a4f-a2f7-485d-a5e5-b3fe75c265e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:23.476887 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac935a4f-a2f7-485d-a5e5-b3fe75c265e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:23.479045 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac935a4f-a2f7-485d-a5e5-b3fe75c265e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:23.532802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac935a4f-a2f7-485d-a5e5-b3fe75c265e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:23.535190 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac935a4f-a2f7-485d-a5e5-b3fe75c265e9 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:23.578736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 893/1788] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:23 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:25.072484 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc531e8-b188-4ec1-bbf3-0788deb3d62f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:25.074818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8dc531e8-b188-4ec1-bbf3-0788deb3d62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:25.107687 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc531e8-b188-4ec1-bbf3-0788deb3d62f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:25.140243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f8b8d-0416-42a5-968f-044cdf8bffe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:25.142484 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f8b8d-0416-42a5-968f-044cdf8bffe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:25.143246 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f8b8d-0416-42a5-968f-044cdf8bffe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:25.145067 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f8b8d-0416-42a5-968f-044cdf8bffe0 admin admin] PATH_INFO: `/v3/users/f2cbb48facb3451b853c4dcb44b20cd2` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:25.153500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f8b8d-0416-42a5-968f-044cdf8bffe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2cbb48facb3451b853c4dcb44b20cd2)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:25.165701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f8b8d-0416-42a5-968f-044cdf8bffe0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:25.192867 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef51087-6c43-4226-a577-57b1e00126dc None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:25.193855 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef51087-6c43-4226-a577-57b1e00126dc None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:25.194298 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef51087-6c43-4226-a577-57b1e00126dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:25.262647 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:25.276005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-6ef51087-6c43-4226-a577-57b1e00126dc None None] MFA Rules not processed for user `c9633b9cbd8a4332a605ca66d73dc5b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:25.278545 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-6ef51087-6c43-4226-a577-57b1e00126dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:25.445906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1e0f8b8d-0416-42a5-968f-044cdf8bffe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2cbb48facb3451b853c4dcb44b20cd2'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:25.489078 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1e0f8b8d-0416-42a5-968f-044cdf8bffe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2cbb48facb3451b853c4dcb44b20cd2'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:25.490785 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1e0f8b8d-0416-42a5-968f-044cdf8bffe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2cbb48facb3451b853c4dcb44b20cd2'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:25.523947 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1e0f8b8d-0416-42a5-968f-044cdf8bffe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2cbb48facb3451b853c4dcb44b20cd2'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:25.525820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1e0f8b8d-0416-42a5-968f-044cdf8bffe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2cbb48facb3451b853c4dcb44b20cd2'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:25.572300 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 896/1789] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:11:25 2022] DELETE /identity/v3/users/f2cbb48facb3451b853c4dcb44b20cd2 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:25.577080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84362e23-c690-4f41-a624-132c02f28acd None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:25.577675 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84362e23-c690-4f41-a624-132c02f28acd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:25.578521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84362e23-c690-4f41-a624-132c02f28acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:25.626721 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-84362e23-c690-4f41-a624-132c02f28acd None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:25.629257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-84362e23-c690-4f41-a624-132c02f28acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:25.640921 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 894/1790] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:25.751607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc00ed-b970-47ef-9feb-c6534a0c9756 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:25.751899 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc00ed-b970-47ef-9feb-c6534a0c9756 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:25.752426 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc00ed-b970-47ef-9feb-c6534a0c9756 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:25.754713 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 895/1791] 10.222.0.100 () {54 vars in 926 bytes} [Tue Aug 2 09:11:25 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:11:25.789705 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc00ed-b970-47ef-9feb-c6534a0c9756 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:25.791621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-72fc00ed-b970-47ef-9feb-c6534a0c9756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:26.018976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 897/1792] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 443 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:26.066294 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:26.137068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc00ed-b970-47ef-9feb-c6534a0c9756 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:26.160645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac86999-3cdb-4fd5-9f6f-94aab86de73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:26.164464 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac86999-3cdb-4fd5-9f6f-94aab86de73d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:26.167032 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac86999-3cdb-4fd5-9f6f-94aab86de73d admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:26.169052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac86999-3cdb-4fd5-9f6f-94aab86de73d admin admin] PATH_INFO: `/v3/users/5e589f8ba5cf4d3b99c3272e35e8d50d` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:26.187161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac86999-3cdb-4fd5-9f6f-94aab86de73d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e589f8ba5cf4d3b99c3272e35e8d50d)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:26.201850 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac86999-3cdb-4fd5-9f6f-94aab86de73d admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:26.363033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84362e23-c690-4f41-a624-132c02f28acd None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:26.366844 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-84362e23-c690-4f41-a624-132c02f28acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:26.429264 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-9ac86999-3cdb-4fd5-9f6f-94aab86de73d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e589f8ba5cf4d3b99c3272e35e8d50d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:26.430703 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-9ac86999-3cdb-4fd5-9f6f-94aab86de73d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e589f8ba5cf4d3b99c3272e35e8d50d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:26.468913 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-9ac86999-3cdb-4fd5-9f6f-94aab86de73d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e589f8ba5cf4d3b99c3272e35e8d50d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:26.475655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84362e23-c690-4f41-a624-132c02f28acd None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:26.514083 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42106de1-fafb-47cc-a665-10aedd59324a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:26.525373 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42106de1-fafb-47cc-a665-10aedd59324a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:26.526997 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42106de1-fafb-47cc-a665-10aedd59324a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:26.528509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42106de1-fafb-47cc-a665-10aedd59324a admin admin] PATH_INFO: `/v3/projects/9b89fc0977074abda8fb75608df14d2b` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:26.537976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42106de1-fafb-47cc-a665-10aedd59324a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b89fc0977074abda8fb75608df14d2b)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:26.549344 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42106de1-fafb-47cc-a665-10aedd59324a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:26.553847 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-9ac86999-3cdb-4fd5-9f6f-94aab86de73d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e589f8ba5cf4d3b99c3272e35e8d50d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:26.595857 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-9ac86999-3cdb-4fd5-9f6f-94aab86de73d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e589f8ba5cf4d3b99c3272e35e8d50d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:26.640858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 896/1793] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:11:25 2022] DELETE /identity/v3/users/5e589f8ba5cf4d3b99c3272e35e8d50d => generated 0 bytes in 847 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:26.646032 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871e815-213f-4eae-ab9f-fac72e0231bf None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:26.646326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871e815-213f-4eae-ab9f-fac72e0231bf None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:26.652045 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871e815-213f-4eae-ab9f-fac72e0231bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:26.732677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-4871e815-213f-4eae-ab9f-fac72e0231bf None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:26.737187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-4871e815-213f-4eae-ab9f-fac72e0231bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:26.750583 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-42106de1-fafb-47cc-a665-10aedd59324a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b89fc0977074abda8fb75608df14d2b', 'request_id': 'req-42106de1-fafb-47cc-a665-10aedd59324a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:26.769821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-42106de1-fafb-47cc-a665-10aedd59324a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b89fc0977074abda8fb75608df14d2b', 'request_id': 'req-42106de1-fafb-47cc-a665-10aedd59324a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:26.786772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-42106de1-fafb-47cc-a665-10aedd59324a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b89fc0977074abda8fb75608df14d2b', 'request_id': 'req-42106de1-fafb-47cc-a665-10aedd59324a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:26.818349 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-42106de1-fafb-47cc-a665-10aedd59324a admin admin] The token cache is being invalidate because project 9b89fc0977074abda8fb75608df14d2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:26.818726 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-42106de1-fafb-47cc-a665-10aedd59324a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:26.819892 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-42106de1-fafb-47cc-a665-10aedd59324a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:26.839721 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 898/1794] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:11:26 2022] DELETE /identity/v3/projects/9b89fc0977074abda8fb75608df14d2b => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:26.844082 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ba253-5bd4-47bb-ad6e-cf55495a6522 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:26.844400 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ba253-5bd4-47bb-ad6e-cf55495a6522 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:26.844797 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ba253-5bd4-47bb-ad6e-cf55495a6522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:26.972314 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 897/1795] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:26.976961 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871e815-213f-4eae-ab9f-fac72e0231bf None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:26.981813 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-989ba253-5bd4-47bb-ad6e-cf55495a6522 None None] MFA Rules not processed for user `49a0c1a4a6cf43e28ec1b8488c681848`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:26.982793 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-4871e815-213f-4eae-ab9f-fac72e0231bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:26.983914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-989ba253-5bd4-47bb-ad6e-cf55495a6522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:27.056361 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:27.151691 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871e815-213f-4eae-ab9f-fac72e0231bf None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:27.177826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811f73a-a495-4897-9c3d-714c17d0caf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:27.178912 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811f73a-a495-4897-9c3d-714c17d0caf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:27.179430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811f73a-a495-4897-9c3d-714c17d0caf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:27.179925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811f73a-a495-4897-9c3d-714c17d0caf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:27.194288 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-0811f73a-a495-4897-9c3d-714c17d0caf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:27.218066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811f73a-a495-4897-9c3d-714c17d0caf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:27.229633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811f73a-a495-4897-9c3d-714c17d0caf1 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:27.257247 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 898/1796] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:26 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:27.258486 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811f73a-a495-4897-9c3d-714c17d0caf1 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:27.260292 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-0811f73a-a495-4897-9c3d-714c17d0caf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:27.275657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811f73a-a495-4897-9c3d-714c17d0caf1 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:27.282100 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b9952-5bd9-4289-8427-cf11f0ce6d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:27.291736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b9952-5bd9-4289-8427-cf11f0ce6d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:27.292320 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b9952-5bd9-4289-8427-cf11f0ce6d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:27.292819 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b9952-5bd9-4289-8427-cf11f0ce6d29 admin admin] PATH_INFO: `/v3/users/b0d47787b92945b7a9474f8e1470e3e3` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:27.294732 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b9952-5bd9-4289-8427-cf11f0ce6d29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0d47787b92945b7a9474f8e1470e3e3)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:27.297251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b9952-5bd9-4289-8427-cf11f0ce6d29 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:27.303952 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:27.397597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 899/1797] 10.222.0.100 () {58 vars in 1023 bytes} [Tue Aug 2 09:11:26 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 557 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:27.398553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ba253-5bd4-47bb-ad6e-cf55495a6522 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:27.406701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-989ba253-5bd4-47bb-ad6e-cf55495a6522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:27.491666 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-720b9952-5bd9-4289-8427-cf11f0ce6d29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0d47787b92945b7a9474f8e1470e3e3'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:27.491909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ba253-5bd4-47bb-ad6e-cf55495a6522 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:27.495696 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-720b9952-5bd9-4289-8427-cf11f0ce6d29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0d47787b92945b7a9474f8e1470e3e3'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:27.511400 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-720b9952-5bd9-4289-8427-cf11f0ce6d29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0d47787b92945b7a9474f8e1470e3e3'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:27.513871 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418cfc6e-492c-4ce5-b41c-066414e0cddd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:27.514524 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cfc6e-492c-4ce5-b41c-066414e0cddd service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:27.530527 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cfc6e-492c-4ce5-b41c-066414e0cddd service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:27.530818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cfc6e-492c-4ce5-b41c-066414e0cddd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:27.538895 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-720b9952-5bd9-4289-8427-cf11f0ce6d29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0d47787b92945b7a9474f8e1470e3e3'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:27.564645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-720b9952-5bd9-4289-8427-cf11f0ce6d29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0d47787b92945b7a9474f8e1470e3e3'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:27.571765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418cfc6e-492c-4ce5-b41c-066414e0cddd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:27.573291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418cfc6e-492c-4ce5-b41c-066414e0cddd service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:27.589223 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 899/1798] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:11:27 2022] DELETE /identity/v3/users/b0d47787b92945b7a9474f8e1470e3e3 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:27.613755 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 900/1799] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:27 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:28.987027 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b9952-5bd9-4289-8427-cf11f0ce6d29 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:28.990043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-720b9952-5bd9-4289-8427-cf11f0ce6d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:29.207714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b9952-5bd9-4289-8427-cf11f0ce6d29 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:29.227555 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba464d7-9873-477a-bbf2-d0a91e64a468 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:29.228462 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba464d7-9873-477a-bbf2-d0a91e64a468 service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:29.229624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba464d7-9873-477a-bbf2-d0a91e64a468 service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:29.230112 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba464d7-9873-477a-bbf2-d0a91e64a468 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:29.232395 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-7ba464d7-9873-477a-bbf2-d0a91e64a468 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:29.303879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba464d7-9873-477a-bbf2-d0a91e64a468 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:29.307946 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba464d7-9873-477a-bbf2-d0a91e64a468 service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:29.353373 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 900/1800] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:28 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:29.379091 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418cfc6e-492c-4ce5-b41c-066414e0cddd service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:29.380928 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-418cfc6e-492c-4ce5-b41c-066414e0cddd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:29.422466 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418cfc6e-492c-4ce5-b41c-066414e0cddd service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:29.556263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b911501-ba32-41ee-a0b2-808f655d8263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:29.557147 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b911501-ba32-41ee-a0b2-808f655d8263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:29.557586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b911501-ba32-41ee-a0b2-808f655d8263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:29.558142 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b911501-ba32-41ee-a0b2-808f655d8263 admin admin] PATH_INFO: `/v3/projects/19f61c15161a40719159ed7579d62fa9` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:29.559884 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b911501-ba32-41ee-a0b2-808f655d8263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19f61c15161a40719159ed7579d62fa9)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:29.563363 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b911501-ba32-41ee-a0b2-808f655d8263 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:29.670508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2b911501-ba32-41ee-a0b2-808f655d8263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19f61c15161a40719159ed7579d62fa9', 'request_id': 'req-2b911501-ba32-41ee-a0b2-808f655d8263'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:29.690069 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2b911501-ba32-41ee-a0b2-808f655d8263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19f61c15161a40719159ed7579d62fa9', 'request_id': 'req-2b911501-ba32-41ee-a0b2-808f655d8263'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:29.710150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2b911501-ba32-41ee-a0b2-808f655d8263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19f61c15161a40719159ed7579d62fa9', 'request_id': 'req-2b911501-ba32-41ee-a0b2-808f655d8263'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:29.739826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2b911501-ba32-41ee-a0b2-808f655d8263 admin admin] The token cache is being invalidate because project 19f61c15161a40719159ed7579d62fa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:29.742616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2b911501-ba32-41ee-a0b2-808f655d8263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:29.744815 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2b911501-ba32-41ee-a0b2-808f655d8263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:29.752153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 901/1801] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:11:29 2022] DELETE /identity/v3/projects/19f61c15161a40719159ed7579d62fa9 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:11:29.757258 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba464d7-9873-477a-bbf2-d0a91e64a468 service nova] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:29.759142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-7ba464d7-9873-477a-bbf2-d0a91e64a468 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:29.802699 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba464d7-9873-477a-bbf2-d0a91e64a468 service nova] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:29.978177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087a9f7-6761-4ad6-8569-22659cbc7da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:29.979632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087a9f7-6761-4ad6-8569-22659cbc7da1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:29.982813 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087a9f7-6761-4ad6-8569-22659cbc7da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:29.982813 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087a9f7-6761-4ad6-8569-22659cbc7da1 admin admin] PATH_INFO: `/v3/users/cf1f6180d0ca49b6b5afc5249b8cee21` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:29.985434 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3087a9f7-6761-4ad6-8569-22659cbc7da1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf1f6180d0ca49b6b5afc5249b8cee21)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:30.001243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3087a9f7-6761-4ad6-8569-22659cbc7da1 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:30.035006 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:30.186151 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3087a9f7-6761-4ad6-8569-22659cbc7da1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf1f6180d0ca49b6b5afc5249b8cee21'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:30.188533 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3087a9f7-6761-4ad6-8569-22659cbc7da1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf1f6180d0ca49b6b5afc5249b8cee21'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:30.216336 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3087a9f7-6761-4ad6-8569-22659cbc7da1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf1f6180d0ca49b6b5afc5249b8cee21'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:30.244479 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3087a9f7-6761-4ad6-8569-22659cbc7da1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf1f6180d0ca49b6b5afc5249b8cee21'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:30.256742 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3087a9f7-6761-4ad6-8569-22659cbc7da1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf1f6180d0ca49b6b5afc5249b8cee21'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:30.276798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 901/1802] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:11:29 2022] DELETE /identity/v3/users/cf1f6180d0ca49b6b5afc5249b8cee21 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:30.974274 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b911501-ba32-41ee-a0b2-808f655d8263 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:30.977074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2b911501-ba32-41ee-a0b2-808f655d8263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:31.017820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b911501-ba32-41ee-a0b2-808f655d8263 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:31.080540 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:31.193146 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9b677-e92b-4838-8f84-1eda9b967380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:31.194054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9b677-e92b-4838-8f84-1eda9b967380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:31.194054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9b677-e92b-4838-8f84-1eda9b967380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:31.197136 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9b677-e92b-4838-8f84-1eda9b967380 admin admin] PATH_INFO: `/v3/projects/71f3d2361688490ea2edc2c1e6ca7664` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:31.200550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9b677-e92b-4838-8f84-1eda9b967380 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71f3d2361688490ea2edc2c1e6ca7664)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:31.205781 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9b677-e92b-4838-8f84-1eda9b967380 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:31.371901 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8bf9b677-e92b-4838-8f84-1eda9b967380 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71f3d2361688490ea2edc2c1e6ca7664', 'request_id': 'req-8bf9b677-e92b-4838-8f84-1eda9b967380'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:31.394742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8bf9b677-e92b-4838-8f84-1eda9b967380 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71f3d2361688490ea2edc2c1e6ca7664', 'request_id': 'req-8bf9b677-e92b-4838-8f84-1eda9b967380'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:31.412770 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8bf9b677-e92b-4838-8f84-1eda9b967380 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71f3d2361688490ea2edc2c1e6ca7664', 'request_id': 'req-8bf9b677-e92b-4838-8f84-1eda9b967380'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:31.446008 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8bf9b677-e92b-4838-8f84-1eda9b967380 admin admin] The token cache is being invalidate because project 71f3d2361688490ea2edc2c1e6ca7664 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:31.446416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8bf9b677-e92b-4838-8f84-1eda9b967380 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:31.452845 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8bf9b677-e92b-4838-8f84-1eda9b967380 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:31.467662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 902/1803] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:11:30 2022] DELETE /identity/v3/projects/71f3d2361688490ea2edc2c1e6ca7664 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:31.471988 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087a9f7-6761-4ad6-8569-22659cbc7da1 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:31.474067 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3087a9f7-6761-4ad6-8569-22659cbc7da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:31.505609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087a9f7-6761-4ad6-8569-22659cbc7da1 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:31.822634 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56deb0-e2a4-49ff-b251-343f518b294a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:31.826168 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56deb0-e2a4-49ff-b251-343f518b294a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:31.827179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56deb0-e2a4-49ff-b251-343f518b294a admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:31.828947 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56deb0-e2a4-49ff-b251-343f518b294a admin admin] PATH_INFO: `/v3/users/19a046e442ad4affa5837c10bc1077d1` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:31.833810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56deb0-e2a4-49ff-b251-343f518b294a admin admin] RBAC: Authorizing `identity:delete_user(user_id=19a046e442ad4affa5837c10bc1077d1)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:31.842571 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56deb0-e2a4-49ff-b251-343f518b294a admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:32.098649 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ce56deb0-e2a4-49ff-b251-343f518b294a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19a046e442ad4affa5837c10bc1077d1'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:32.108174 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ce56deb0-e2a4-49ff-b251-343f518b294a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19a046e442ad4affa5837c10bc1077d1'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:32.147532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ce56deb0-e2a4-49ff-b251-343f518b294a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19a046e442ad4affa5837c10bc1077d1'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:32.217449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ce56deb0-e2a4-49ff-b251-343f518b294a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19a046e442ad4affa5837c10bc1077d1'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:32.246599 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ce56deb0-e2a4-49ff-b251-343f518b294a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19a046e442ad4affa5837c10bc1077d1'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:32.261918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 902/1804] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:11:31 2022] DELETE /identity/v3/users/19a046e442ad4affa5837c10bc1077d1 => generated 0 bytes in 788 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:33.140254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9b677-e92b-4838-8f84-1eda9b967380 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:33.142898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8bf9b677-e92b-4838-8f84-1eda9b967380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:33.165328 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56deb0-e2a4-49ff-b251-343f518b294a admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:33.170054 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ce56deb0-e2a4-49ff-b251-343f518b294a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:33.173236 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9b677-e92b-4838-8f84-1eda9b967380 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:33.234306 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:33.336170 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a2087d-6bc0-4e0c-a751-c010d80b1b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:33.336934 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2087d-6bc0-4e0c-a751-c010d80b1b98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:33.337335 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2087d-6bc0-4e0c-a751-c010d80b1b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:33.337681 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2087d-6bc0-4e0c-a751-c010d80b1b98 admin admin] PATH_INFO: `/v3/projects/4382a8dd6a5e4c89b54055f1708be577` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:33.341523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2087d-6bc0-4e0c-a751-c010d80b1b98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4382a8dd6a5e4c89b54055f1708be577)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:33.343722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2087d-6bc0-4e0c-a751-c010d80b1b98 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:33.414028 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56deb0-e2a4-49ff-b251-343f518b294a admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:33.433334 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07be04-2a94-4aeb-b832-16ed03605d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:33.434341 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07be04-2a94-4aeb-b832-16ed03605d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:33.434638 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07be04-2a94-4aeb-b832-16ed03605d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:33.435801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07be04-2a94-4aeb-b832-16ed03605d11 admin admin] PATH_INFO: `/v3/projects/cafcb6d6ac6f4d39b0b8ebc2ad891934` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:33.442080 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f07be04-2a94-4aeb-b832-16ed03605d11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cafcb6d6ac6f4d39b0b8ebc2ad891934)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:33.444644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f07be04-2a94-4aeb-b832-16ed03605d11 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:33.463590 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-80a2087d-6bc0-4e0c-a751-c010d80b1b98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4382a8dd6a5e4c89b54055f1708be577', 'request_id': 'req-80a2087d-6bc0-4e0c-a751-c010d80b1b98'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:33.486157 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-80a2087d-6bc0-4e0c-a751-c010d80b1b98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4382a8dd6a5e4c89b54055f1708be577', 'request_id': 'req-80a2087d-6bc0-4e0c-a751-c010d80b1b98'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:33.498961 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:33.508939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-80a2087d-6bc0-4e0c-a751-c010d80b1b98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4382a8dd6a5e4c89b54055f1708be577', 'request_id': 'req-80a2087d-6bc0-4e0c-a751-c010d80b1b98'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:33.529896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-80a2087d-6bc0-4e0c-a751-c010d80b1b98 admin admin] The token cache is being invalidate because project 4382a8dd6a5e4c89b54055f1708be577 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:33.530199 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-80a2087d-6bc0-4e0c-a751-c010d80b1b98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:33.543039 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-80a2087d-6bc0-4e0c-a751-c010d80b1b98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:33.548205 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 903/1805] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:11:33 2022] DELETE /identity/v3/projects/4382a8dd6a5e4c89b54055f1708be577 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:33.553352 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c25f0b-c32d-447b-ab3e-d9ec1f8b0721 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:33.553645 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c25f0b-c32d-447b-ab3e-d9ec1f8b0721 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:33.553928 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c25f0b-c32d-447b-ab3e-d9ec1f8b0721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:33.604976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f7c25f0b-c32d-447b-ab3e-d9ec1f8b0721 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:33.606240 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f7c25f0b-c32d-447b-ab3e-d9ec1f8b0721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:33.663011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5f07be04-2a94-4aeb-b832-16ed03605d11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cafcb6d6ac6f4d39b0b8ebc2ad891934', 'request_id': 'req-5f07be04-2a94-4aeb-b832-16ed03605d11'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:33.687251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5f07be04-2a94-4aeb-b832-16ed03605d11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cafcb6d6ac6f4d39b0b8ebc2ad891934', 'request_id': 'req-5f07be04-2a94-4aeb-b832-16ed03605d11'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:33.707635 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5f07be04-2a94-4aeb-b832-16ed03605d11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cafcb6d6ac6f4d39b0b8ebc2ad891934', 'request_id': 'req-5f07be04-2a94-4aeb-b832-16ed03605d11'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:33.778697 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5f07be04-2a94-4aeb-b832-16ed03605d11 admin admin] The token cache is being invalidate because project cafcb6d6ac6f4d39b0b8ebc2ad891934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:33.779002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5f07be04-2a94-4aeb-b832-16ed03605d11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:33.781570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5f07be04-2a94-4aeb-b832-16ed03605d11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:33.783755 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 903/1806] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:11:33 2022] DELETE /identity/v3/projects/cafcb6d6ac6f4d39b0b8ebc2ad891934 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:33.793560 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9825d-0afa-4e81-84e6-65a166f98abb None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:33.793848 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9825d-0afa-4e81-84e6-65a166f98abb None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:33.794144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9825d-0afa-4e81-84e6-65a166f98abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:33.860041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 904/1807] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:33.887330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977e7fb-df6e-4133-837b-ff62f1608532 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:33.887725 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977e7fb-df6e-4133-837b-ff62f1608532 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:33.888012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977e7fb-df6e-4133-837b-ff62f1608532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:33.890240 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-39a9825d-0afa-4e81-84e6-65a166f98abb None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:33.891591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-39a9825d-0afa-4e81-84e6-65a166f98abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:33.937917 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 904/1808] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:33.947881 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ae08f-0cf3-4ae9-9d48-eae9cce3f5c3 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:33.949365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ae08f-0cf3-4ae9-9d48-eae9cce3f5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:33.949894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ae08f-0cf3-4ae9-9d48-eae9cce3f5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:33.954381 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b977e7fb-df6e-4133-837b-ff62f1608532 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:33.956204 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b977e7fb-df6e-4133-837b-ff62f1608532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:33.982075 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 905/1809] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:11:33.986392 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977e7fb-df6e-4133-837b-ff62f1608532 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:34.015804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977e7fb-df6e-4133-837b-ff62f1608532 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:34.030195 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136b2b3-d2d2-49c1-a4ac-782a91361106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:34.031045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136b2b3-d2d2-49c1-a4ac-782a91361106 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:34.031581 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136b2b3-d2d2-49c1-a4ac-782a91361106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:34.032026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136b2b3-d2d2-49c1-a4ac-782a91361106 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:34.032891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136b2b3-d2d2-49c1-a4ac-782a91361106 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:11:34.033362 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136b2b3-d2d2-49c1-a4ac-782a91361106 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:34.036436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136b2b3-d2d2-49c1-a4ac-782a91361106 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:34.083643 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-c82ae08f-0cf3-4ae9-9d48-eae9cce3f5c3 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:34.093281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c82ae08f-0cf3-4ae9-9d48-eae9cce3f5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:34.123494 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 906/1810] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:11:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:11:34.130594 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136b2b3-d2d2-49c1-a4ac-782a91361106 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:34.160330 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 905/1811] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:34.171726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ae08f-0cf3-4ae9-9d48-eae9cce3f5c3 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:34.172810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136b2b3-d2d2-49c1-a4ac-782a91361106 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:34.197545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746ede4-f996-4881-98ca-c91f53e706df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:34.198187 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746ede4-f996-4881-98ca-c91f53e706df admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:34.198638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746ede4-f996-4881-98ca-c91f53e706df admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:34.199061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746ede4-f996-4881-98ca-c91f53e706df admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:34.200040 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746ede4-f996-4881-98ca-c91f53e706df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:34.202149 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746ede4-f996-4881-98ca-c91f53e706df admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:34.241920 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ae08f-0cf3-4ae9-9d48-eae9cce3f5c3 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:34.279993 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdae264-9ed9-4599-89e9-53075bf6a926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:34.288766 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdae264-9ed9-4599-89e9-53075bf6a926 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:34.292047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdae264-9ed9-4599-89e9-53075bf6a926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:34.293091 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdae264-9ed9-4599-89e9-53075bf6a926 admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:34.299903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdae264-9ed9-4599-89e9-53075bf6a926 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:11:34.302646 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdae264-9ed9-4599-89e9-53075bf6a926 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:34.312124 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdae264-9ed9-4599-89e9-53075bf6a926 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:34.374868 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 907/1812] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:11:34 2022] POST /identity/v3/projects => generated 378 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:34.379874 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 906/1813] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:11:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:11:34.382865 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746ede4-f996-4881-98ca-c91f53e706df admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:34.383104 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdae264-9ed9-4599-89e9-53075bf6a926 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:34.446773 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdae264-9ed9-4599-89e9-53075bf6a926 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:34.552432 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746ede4-f996-4881-98ca-c91f53e706df admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:34.558385 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00578b-a7b8-41b3-9f03-ed7560e1b964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:34.559499 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00578b-a7b8-41b3-9f03-ed7560e1b964 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:34.559867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00578b-a7b8-41b3-9f03-ed7560e1b964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:34.560204 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00578b-a7b8-41b3-9f03-ed7560e1b964 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:34.561906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a00578b-a7b8-41b3-9f03-ed7560e1b964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:34.564630 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a00578b-a7b8-41b3-9f03-ed7560e1b964 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:34.604854 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452e10e-e108-4eb3-8a93-6f7b0dbf45f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:34.606410 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452e10e-e108-4eb3-8a93-6f7b0dbf45f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:34.606825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452e10e-e108-4eb3-8a93-6f7b0dbf45f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:34.607254 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452e10e-e108-4eb3-8a93-6f7b0dbf45f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:34.608433 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d452e10e-e108-4eb3-8a93-6f7b0dbf45f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:34.612612 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d452e10e-e108-4eb3-8a93-6f7b0dbf45f7 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:34.620729 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 908/1814] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:11:34 2022] POST /identity/v3/users => generated 290 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:34.622664 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00578b-a7b8-41b3-9f03-ed7560e1b964 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:34.634647 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8a00578b-a7b8-41b3-9f03-ed7560e1b964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:34.664704 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00578b-a7b8-41b3-9f03-ed7560e1b964 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:34.682207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7e1ce-6c1f-4537-ad33-062b66f3bb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:34.684860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7e1ce-6c1f-4537-ad33-062b66f3bb11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:34.685715 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7e1ce-6c1f-4537-ad33-062b66f3bb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:34.686977 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7e1ce-6c1f-4537-ad33-062b66f3bb11 admin admin] PATH_INFO: `/v3/projects/f2dd1fe506ae41c5a8192bdee184ee44` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:34.693164 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7e1ce-6c1f-4537-ad33-062b66f3bb11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2dd1fe506ae41c5a8192bdee184ee44)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:34.705893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7e1ce-6c1f-4537-ad33-062b66f3bb11 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:34.760061 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:34.801685 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 907/1815] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:11:34 2022] POST /identity/v3/projects => generated 368 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:34.803165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452e10e-e108-4eb3-8a93-6f7b0dbf45f7 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:34.811696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:34.863380 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fdb7e1ce-6c1f-4537-ad33-062b66f3bb11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2dd1fe506ae41c5a8192bdee184ee44', 'request_id': 'req-fdb7e1ce-6c1f-4537-ad33-062b66f3bb11'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:34.916652 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fdb7e1ce-6c1f-4537-ad33-062b66f3bb11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2dd1fe506ae41c5a8192bdee184ee44', 'request_id': 'req-fdb7e1ce-6c1f-4537-ad33-062b66f3bb11'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:34.947179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fdb7e1ce-6c1f-4537-ad33-062b66f3bb11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2dd1fe506ae41c5a8192bdee184ee44', 'request_id': 'req-fdb7e1ce-6c1f-4537-ad33-062b66f3bb11'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:34.976384 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452e10e-e108-4eb3-8a93-6f7b0dbf45f7 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:34.980008 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fdb7e1ce-6c1f-4537-ad33-062b66f3bb11 admin admin] The token cache is being invalidate because project f2dd1fe506ae41c5a8192bdee184ee44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:34.980812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fdb7e1ce-6c1f-4537-ad33-062b66f3bb11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:34.983819 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fdb7e1ce-6c1f-4537-ad33-062b66f3bb11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:34.991561 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 909/1816] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:11:34 2022] DELETE /identity/v3/projects/f2dd1fe506ae41c5a8192bdee184ee44 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:34.992773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7e1ce-6c1f-4537-ad33-062b66f3bb11 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:34.994781 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fdb7e1ce-6c1f-4537-ad33-062b66f3bb11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:35.113876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5a928-cfce-450f-8e05-0b164c69f14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:35.116248 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5a928-cfce-450f-8e05-0b164c69f14a admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:35.116694 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5a928-cfce-450f-8e05-0b164c69f14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:35.117227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5a928-cfce-450f-8e05-0b164c69f14a admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:35.118088 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5a928-cfce-450f-8e05-0b164c69f14a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:35.121427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5a928-cfce-450f-8e05-0b164c69f14a admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:35.142774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7e1ce-6c1f-4537-ad33-062b66f3bb11 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:35.145075 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 908/1817] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:34 2022] GET /identity/v3/roles => generated 1427 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:35.147260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784acc8-1246-4680-bb3e-13a05d9f2030 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:35.148142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784acc8-1246-4680-bb3e-13a05d9f2030 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:35.149013 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784acc8-1246-4680-bb3e-13a05d9f2030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:35.159792 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde0a44-e0eb-4c49-ad25-afd2d22c0381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:35.160568 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde0a44-e0eb-4c49-ad25-afd2d22c0381 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:35.161521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde0a44-e0eb-4c49-ad25-afd2d22c0381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:35.162752 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde0a44-e0eb-4c49-ad25-afd2d22c0381 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:35.163475 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde0a44-e0eb-4c49-ad25-afd2d22c0381 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:35.167301 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde0a44-e0eb-4c49-ad25-afd2d22c0381 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:35.199536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-b784acc8-1246-4680-bb3e-13a05d9f2030 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:35.202376 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b784acc8-1246-4680-bb3e-13a05d9f2030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:35.219738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 910/1818] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:11:34 2022] POST /identity/v3/users => generated 285 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:35.223629 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde0a44-e0eb-4c49-ad25-afd2d22c0381 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:35.227947 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 909/1819] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:35.228884 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7bde0a44-e0eb-4c49-ad25-afd2d22c0381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:35.231471 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784acc8-1246-4680-bb3e-13a05d9f2030 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:35.253747 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784acc8-1246-4680-bb3e-13a05d9f2030 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:35.259196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde0a44-e0eb-4c49-ad25-afd2d22c0381 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:35.260892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f2ae7-7265-4f50-b106-5f0f3bec1e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:35.262631 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f2ae7-7265-4f50-b106-5f0f3bec1e5c admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:35.262953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f2ae7-7265-4f50-b106-5f0f3bec1e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:35.263239 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f2ae7-7265-4f50-b106-5f0f3bec1e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:35.264115 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f2ae7-7265-4f50-b106-5f0f3bec1e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:35.266118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f2ae7-7265-4f50-b106-5f0f3bec1e5c admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:35.267615 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791a5ce-d2a1-4b9c-82c3-024f3e48935f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:35.269393 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791a5ce-d2a1-4b9c-82c3-024f3e48935f admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:35.269881 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791a5ce-d2a1-4b9c-82c3-024f3e48935f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:35.270382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791a5ce-d2a1-4b9c-82c3-024f3e48935f admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:35.271461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791a5ce-d2a1-4b9c-82c3-024f3e48935f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:35.274040 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791a5ce-d2a1-4b9c-82c3-024f3e48935f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:35.281377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 910/1820] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:35 2022] GET /identity/v3/roles => generated 1427 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:35.281377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067452c-c33b-4191-8efc-f9e62cee9695 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:35.281377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067452c-c33b-4191-8efc-f9e62cee9695 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:35.281377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067452c-c33b-4191-8efc-f9e62cee9695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:35.295085 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 911/1821] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:35 2022] GET /identity/v3/roles => generated 1427 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:35.296330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791a5ce-d2a1-4b9c-82c3-024f3e48935f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:35.306247 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-2067452c-c33b-4191-8efc-f9e62cee9695 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:35.307483 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-2067452c-c33b-4191-8efc-f9e62cee9695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:35.309824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791a5ce-d2a1-4b9c-82c3-024f3e48935f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:35.317150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 911/1822] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:35.317741 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067452c-c33b-4191-8efc-f9e62cee9695 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:35.318197 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1e95a-5aa1-4561-8e43-9c06ebdfe7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:35.319028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1e95a-5aa1-4561-8e43-9c06ebdfe7f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:35.319612 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1e95a-5aa1-4561-8e43-9c06ebdfe7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:35.320082 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1e95a-5aa1-4561-8e43-9c06ebdfe7f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:35.322400 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1e95a-5aa1-4561-8e43-9c06ebdfe7f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:35.324456 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1e95a-5aa1-4561-8e43-9c06ebdfe7f1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:35.332449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067452c-c33b-4191-8efc-f9e62cee9695 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:35.337791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba015a-160d-478e-984a-0d2f20f473b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:35.338629 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba015a-160d-478e-984a-0d2f20f473b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:35.338913 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba015a-160d-478e-984a-0d2f20f473b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:35.339187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba015a-160d-478e-984a-0d2f20f473b3 admin admin] PATH_INFO: `/v3/projects/3c1f7edebdbc471fbef44e7833ef1cef/users/6c75f20ae1dd4984a1f770dbe944ff69/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:35.345536 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 912/1823] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:35 2022] GET /identity/v3/roles => generated 1427 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:35.347139 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1e95a-5aa1-4561-8e43-9c06ebdfe7f1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:35.359455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1e95a-5aa1-4561-8e43-9c06ebdfe7f1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:35.359986 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba015a-160d-478e-984a-0d2f20f473b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c1f7edebdbc471fbef44e7833ef1cef, user_id=6c75f20ae1dd4984a1f770dbe944ff69, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:35.363801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba015a-160d-478e-984a-0d2f20f473b3 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:35.366970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93657e-1157-49d2-be13-d7c21cf63c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:35.369255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93657e-1157-49d2-be13-d7c21cf63c1f admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:35.369734 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93657e-1157-49d2-be13-d7c21cf63c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:35.370369 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93657e-1157-49d2-be13-d7c21cf63c1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:35.371354 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93657e-1157-49d2-be13-d7c21cf63c1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:11:35.371858 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93657e-1157-49d2-be13-d7c21cf63c1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:35.373947 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93657e-1157-49d2-be13-d7c21cf63c1f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:35.385188 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 912/1824] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:35 2022] PUT /identity/v3/projects/3c1f7edebdbc471fbef44e7833ef1cef/users/6c75f20ae1dd4984a1f770dbe944ff69/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:35.386420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba015a-160d-478e-984a-0d2f20f473b3 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:35.391029 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 913/1825] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:11:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:11:35.397183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c184f1c-a7ad-4d89-b0c7-1b4a3ceaa015 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:35.397605 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c184f1c-a7ad-4d89-b0c7-1b4a3ceaa015 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:35.397891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c184f1c-a7ad-4d89-b0c7-1b4a3ceaa015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:35.443454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3c184f1c-a7ad-4d89-b0c7-1b4a3ceaa015 None None] MFA Rules not processed for user `6c75f20ae1dd4984a1f770dbe944ff69`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:35.444944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3c184f1c-a7ad-4d89-b0c7-1b4a3ceaa015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:35.479444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:35.503499 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba015a-160d-478e-984a-0d2f20f473b3 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:35.514601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c40755-5c7f-4220-a51c-4ed15ed3876c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:35.515306 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c40755-5c7f-4220-a51c-4ed15ed3876c admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:35.515591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c40755-5c7f-4220-a51c-4ed15ed3876c admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:35.515869 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c40755-5c7f-4220-a51c-4ed15ed3876c admin admin] PATH_INFO: `/v3/projects/9b165a73dfc44daeaacb2450d21db5e6/users/a538edc2994744de8b12549a8df4892d/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:35.534883 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c40755-5c7f-4220-a51c-4ed15ed3876c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b165a73dfc44daeaacb2450d21db5e6, user_id=a538edc2994744de8b12549a8df4892d, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:35.537933 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c40755-5c7f-4220-a51c-4ed15ed3876c admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:35.557864 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 913/1826] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:35 2022] PUT /identity/v3/projects/9b165a73dfc44daeaacb2450d21db5e6/users/a538edc2994744de8b12549a8df4892d/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:35.558723 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c40755-5c7f-4220-a51c-4ed15ed3876c admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:35.609099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:35.617035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 914/1827] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:35.618572 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213b25e-1704-4e00-beb1-e73b1374d39f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:35.619117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213b25e-1704-4e00-beb1-e73b1374d39f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:35.619969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213b25e-1704-4e00-beb1-e73b1374d39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:35.752196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8213b25e-1704-4e00-beb1-e73b1374d39f None None] MFA Rules not processed for user `a538edc2994744de8b12549a8df4892d`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:35.755890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8213b25e-1704-4e00-beb1-e73b1374d39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:35.789533 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c40755-5c7f-4220-a51c-4ed15ed3876c admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:35.795404 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd993be5-0630-404e-aa50-4a01eb7bbead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:35.796155 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd993be5-0630-404e-aa50-4a01eb7bbead admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:35.796519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd993be5-0630-404e-aa50-4a01eb7bbead admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:35.796866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd993be5-0630-404e-aa50-4a01eb7bbead admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:35.798116 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd993be5-0630-404e-aa50-4a01eb7bbead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:35.803301 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd993be5-0630-404e-aa50-4a01eb7bbead admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:35.869635 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 914/1828] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:11:35 2022] POST /identity/v3/projects => generated 400 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:35.870971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd993be5-0630-404e-aa50-4a01eb7bbead admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:35.885445 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-dd993be5-0630-404e-aa50-4a01eb7bbead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:35.934816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 915/1829] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:35.935684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213b25e-1704-4e00-beb1-e73b1374d39f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:35.960337 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213b25e-1704-4e00-beb1-e73b1374d39f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:35.996756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd993be5-0630-404e-aa50-4a01eb7bbead admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:36.004298 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b111fa-6167-4fa5-a89f-a5681ae78917 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:36.005729 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b111fa-6167-4fa5-a89f-a5681ae78917 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:36.008717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b111fa-6167-4fa5-a89f-a5681ae78917 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:36.009342 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b111fa-6167-4fa5-a89f-a5681ae78917 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:36.123457 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ad88a-20ac-4fc4-8c42-bb44ff01a96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:36.124208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ad88a-20ac-4fc4-8c42-bb44ff01a96d admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:36.124568 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ad88a-20ac-4fc4-8c42-bb44ff01a96d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:36.126678 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ad88a-20ac-4fc4-8c42-bb44ff01a96d admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:36.127936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ad88a-20ac-4fc4-8c42-bb44ff01a96d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:36.129938 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b111fa-6167-4fa5-a89f-a5681ae78917 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:36.130541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ad88a-20ac-4fc4-8c42-bb44ff01a96d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:36.136824 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b111fa-6167-4fa5-a89f-a5681ae78917 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:36.161202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 916/1830] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:11:35 2022] POST /identity/v3/users => generated 301 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:36.161202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ad88a-20ac-4fc4-8c42-bb44ff01a96d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:36.178735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 915/1831] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:35 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:36.181159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b111fa-6167-4fa5-a89f-a5681ae78917 service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:36.181652 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ad88a-20ac-4fc4-8c42-bb44ff01a96d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:36.188203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5347d-f6f4-4693-a383-6887d84c1795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:36.189550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5347d-f6f4-4693-a383-6887d84c1795 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:36.190019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5347d-f6f4-4693-a383-6887d84c1795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:36.190420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5347d-f6f4-4693-a383-6887d84c1795 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:36.191388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5347d-f6f4-4693-a383-6887d84c1795 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:36.193470 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5347d-f6f4-4693-a383-6887d84c1795 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:36.202173 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b111fa-6167-4fa5-a89f-a5681ae78917 service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:36.210974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a7361c-b93b-41e3-af20-9526cc997441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:36.211891 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7361c-b93b-41e3-af20-9526cc997441 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:36.212398 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7361c-b93b-41e3-af20-9526cc997441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:36.212831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7361c-b93b-41e3-af20-9526cc997441 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:36.213917 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a7361c-b93b-41e3-af20-9526cc997441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:36.215784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a7361c-b93b-41e3-af20-9526cc997441 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:36.231668 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 916/1832] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:36 2022] GET /identity/v3/roles => generated 1427 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:36.235295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a7361c-b93b-41e3-af20-9526cc997441 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:36.241899 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:36.246802 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 917/1833] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:11:36 2022] POST /identity/v3/projects => generated 370 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:36.249090 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5347d-f6f4-4693-a383-6887d84c1795 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:36.251568 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a7361c-b93b-41e3-af20-9526cc997441 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:36.333416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:36.339382 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f2b3c-7059-42ff-9fb9-9ef82b34b99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:36.340449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f2b3c-7059-42ff-9fb9-9ef82b34b99f admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:36.340742 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f2b3c-7059-42ff-9fb9-9ef82b34b99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:36.341026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f2b3c-7059-42ff-9fb9-9ef82b34b99f admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:36.341912 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f2b3c-7059-42ff-9fb9-9ef82b34b99f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:36.343653 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5347d-f6f4-4693-a383-6887d84c1795 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:36.344223 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f2b3c-7059-42ff-9fb9-9ef82b34b99f admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:36.355752 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d18227-1940-4718-b2f4-8dea0980ac34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:36.356529 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d18227-1940-4718-b2f4-8dea0980ac34 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:36.356946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d18227-1940-4718-b2f4-8dea0980ac34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:36.357422 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d18227-1940-4718-b2f4-8dea0980ac34 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:36.358756 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d18227-1940-4718-b2f4-8dea0980ac34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:36.360588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d18227-1940-4718-b2f4-8dea0980ac34 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:36.369216 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 917/1834] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:36 2022] GET /identity/v3/roles => generated 1427 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:11:36.370857 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f2b3c-7059-42ff-9fb9-9ef82b34b99f admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:36.394870 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f2b3c-7059-42ff-9fb9-9ef82b34b99f admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:36.397155 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 918/1835] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:11:36 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:36.400919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d18227-1940-4718-b2f4-8dea0980ac34 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:36.404137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb6f68-31f4-4105-bc69-2f08fff494b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:36.405366 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb6f68-31f4-4105-bc69-2f08fff494b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:36.405652 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb6f68-31f4-4105-bc69-2f08fff494b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:36.405929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb6f68-31f4-4105-bc69-2f08fff494b8 admin admin] PATH_INFO: `/v3/projects/34f66feb02ae4e88bb9cbb96fca6f734/users/194bd5ee756f42eba846bee2d1692bfb/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:36.421848 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d18227-1940-4718-b2f4-8dea0980ac34 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:36.432110 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d912df-5250-49da-96e4-f4a70fab84d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:36.432726 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d912df-5250-49da-96e4-f4a70fab84d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:36.433209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d912df-5250-49da-96e4-f4a70fab84d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:36.433735 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d912df-5250-49da-96e4-f4a70fab84d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:36.434569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d912df-5250-49da-96e4-f4a70fab84d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:36.435210 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb6f68-31f4-4105-bc69-2f08fff494b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f66feb02ae4e88bb9cbb96fca6f734, user_id=194bd5ee756f42eba846bee2d1692bfb, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:36.436222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d912df-5250-49da-96e4-f4a70fab84d2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:36.438487 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb6f68-31f4-4105-bc69-2f08fff494b8 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:36.453824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 919/1836] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:36 2022] GET /identity/v3/roles => generated 1427 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:11:36.455881 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 918/1837] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:36 2022] PUT /identity/v3/projects/34f66feb02ae4e88bb9cbb96fca6f734/users/194bd5ee756f42eba846bee2d1692bfb/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:36.457688 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d912df-5250-49da-96e4-f4a70fab84d2 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:36.468768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d984bd8-65c0-43c8-960f-c88e729a078a None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:36.468768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d984bd8-65c0-43c8-960f-c88e729a078a None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:36.468991 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d984bd8-65c0-43c8-960f-c88e729a078a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:36.477933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d912df-5250-49da-96e4-f4a70fab84d2 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:36.508798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-3d984bd8-65c0-43c8-960f-c88e729a078a None None] MFA Rules not processed for user `194bd5ee756f42eba846bee2d1692bfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:36.510467 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3d984bd8-65c0-43c8-960f-c88e729a078a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:36.594363 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5bc5c-1121-4b5b-863d-4e3d52ab63d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:36.595303 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5bc5c-1121-4b5b-863d-4e3d52ab63d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:36.595592 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5bc5c-1121-4b5b-863d-4e3d52ab63d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:36.595878 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5bc5c-1121-4b5b-863d-4e3d52ab63d5 admin admin] PATH_INFO: `/v3/projects/2e0c8030442e4afa926cf27face695d1/users/a32c3e035cf94e08a54e25c2bae31a6f/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:36.618702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5bc5c-1121-4b5b-863d-4e3d52ab63d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e0c8030442e4afa926cf27face695d1, user_id=a32c3e035cf94e08a54e25c2bae31a6f, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:36.622342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5bc5c-1121-4b5b-863d-4e3d52ab63d5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:36.632597 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 919/1838] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:36 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:36.634453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d984bd8-65c0-43c8-960f-c88e729a078a None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:36.644381 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 920/1839] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:36 2022] PUT /identity/v3/projects/2e0c8030442e4afa926cf27face695d1/users/a32c3e035cf94e08a54e25c2bae31a6f/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:36.648930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5bc5c-1121-4b5b-863d-4e3d52ab63d5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:36.684939 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d984bd8-65c0-43c8-960f-c88e729a078a None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:36.808930 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff8957-5e7f-4610-a60e-dcc01a24acf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:36.809633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff8957-5e7f-4610-a60e-dcc01a24acf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:36.809937 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff8957-5e7f-4610-a60e-dcc01a24acf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:36.810194 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff8957-5e7f-4610-a60e-dcc01a24acf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:36.810878 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff8957-5e7f-4610-a60e-dcc01a24acf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:36.813020 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff8957-5e7f-4610-a60e-dcc01a24acf6 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:36.815065 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5bc5c-1121-4b5b-863d-4e3d52ab63d5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:36.824275 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42cea9-c058-4c72-82c4-69f14021e442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:36.824527 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:36.825452 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42cea9-c058-4c72-82c4-69f14021e442 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:36.825816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42cea9-c058-4c72-82c4-69f14021e442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:36.826200 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42cea9-c058-4c72-82c4-69f14021e442 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:36.827381 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42cea9-c058-4c72-82c4-69f14021e442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:36.829283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42cea9-c058-4c72-82c4-69f14021e442 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:36.853056 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 921/1840] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:36 2022] GET /identity/v3/roles => generated 1427 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:36.858623 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42cea9-c058-4c72-82c4-69f14021e442 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:36.869100 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 920/1841] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:11:36 2022] POST /identity/v3/projects => generated 402 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:36.874151 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff8957-5e7f-4610-a60e-dcc01a24acf6 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:36.883256 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42cea9-c058-4c72-82c4-69f14021e442 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:36.962951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:36.973697 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490600e-e4d3-4bf4-8d99-c4b79311c9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:36.974337 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490600e-e4d3-4bf4-8d99-c4b79311c9ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:36.974678 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490600e-e4d3-4bf4-8d99-c4b79311c9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:36.974970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490600e-e4d3-4bf4-8d99-c4b79311c9ff admin admin] PATH_INFO: `/v3/projects/2e0c8030442e4afa926cf27face695d1/users/a32c3e035cf94e08a54e25c2bae31a6f/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:36.977274 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490600e-e4d3-4bf4-8d99-c4b79311c9ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e0c8030442e4afa926cf27face695d1, user_id=a32c3e035cf94e08a54e25c2bae31a6f, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:36.982406 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490600e-e4d3-4bf4-8d99-c4b79311c9ff admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:37.015615 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff8957-5e7f-4610-a60e-dcc01a24acf6 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:37.016963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 922/1842] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:36 2022] PUT /identity/v3/projects/2e0c8030442e4afa926cf27face695d1/users/a32c3e035cf94e08a54e25c2bae31a6f/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:37.020152 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67781b49-0f51-4d3e-86f9-da9058fcc665 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:37.020707 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67781b49-0f51-4d3e-86f9-da9058fcc665 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:37.021336 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67781b49-0f51-4d3e-86f9-da9058fcc665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:37.072863 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-67781b49-0f51-4d3e-86f9-da9058fcc665 None None] MFA Rules not processed for user `a32c3e035cf94e08a54e25c2bae31a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:37.075753 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-67781b49-0f51-4d3e-86f9-da9058fcc665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:37.137945 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea284716-5595-45fc-84cb-94bcbcfd67ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:37.138841 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea284716-5595-45fc-84cb-94bcbcfd67ec admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:37.139317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea284716-5595-45fc-84cb-94bcbcfd67ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:37.139771 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea284716-5595-45fc-84cb-94bcbcfd67ec admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:37.140644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea284716-5595-45fc-84cb-94bcbcfd67ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:37.147908 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea284716-5595-45fc-84cb-94bcbcfd67ec admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:37.184985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 921/1843] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:11:36 2022] POST /identity/v3/users => generated 308 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:37.184985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea284716-5595-45fc-84cb-94bcbcfd67ec admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:37.198840 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea284716-5595-45fc-84cb-94bcbcfd67ec admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:37.215372 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8f67b-f354-4abe-8ffb-de231b777d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:37.217072 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8f67b-f354-4abe-8ffb-de231b777d38 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:37.217602 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8f67b-f354-4abe-8ffb-de231b777d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:37.218221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8f67b-f354-4abe-8ffb-de231b777d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:37.222995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8f67b-f354-4abe-8ffb-de231b777d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:37.224726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8f67b-f354-4abe-8ffb-de231b777d38 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:37.240683 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 922/1844] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:37 2022] GET /identity/v3/roles => generated 1427 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:37.242190 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8f67b-f354-4abe-8ffb-de231b777d38 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:37.256940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 923/1845] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:37.265169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67781b49-0f51-4d3e-86f9-da9058fcc665 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:37.265477 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8f67b-f354-4abe-8ffb-de231b777d38 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:37.267008 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-67781b49-0f51-4d3e-86f9-da9058fcc665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:37.277096 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9f9e3-f8e2-4f73-a7a5-0a13fd9f41b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:37.277096 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9f9e3-f8e2-4f73-a7a5-0a13fd9f41b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:37.277096 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9f9e3-f8e2-4f73-a7a5-0a13fd9f41b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:37.277096 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9f9e3-f8e2-4f73-a7a5-0a13fd9f41b0 admin admin] PATH_INFO: `/v3/projects/e908e08a518642b1a69a0fddc08441e8/users/43aee6558d614ad58770deea1e607b0f/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:37.295507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9f9e3-f8e2-4f73-a7a5-0a13fd9f41b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e908e08a518642b1a69a0fddc08441e8, user_id=43aee6558d614ad58770deea1e607b0f, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:37.298581 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9f9e3-f8e2-4f73-a7a5-0a13fd9f41b0 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:37.316137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 923/1846] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:37 2022] PUT /identity/v3/projects/e908e08a518642b1a69a0fddc08441e8/users/43aee6558d614ad58770deea1e607b0f/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:37.320748 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9f9e3-f8e2-4f73-a7a5-0a13fd9f41b0 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:37.350947 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67781b49-0f51-4d3e-86f9-da9058fcc665 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:37.394883 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:37.427936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9f9e3-f8e2-4f73-a7a5-0a13fd9f41b0 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:37.428168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bbb2c-30c4-4790-af57-19f5fdb8f265 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:37.428837 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bbb2c-30c4-4790-af57-19f5fdb8f265 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:37.429386 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bbb2c-30c4-4790-af57-19f5fdb8f265 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:37.429770 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bbb2c-30c4-4790-af57-19f5fdb8f265 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:37.436933 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57086c7a-9ec7-419e-8f56-060d1036996c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:37.438750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57086c7a-9ec7-419e-8f56-060d1036996c admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:37.439396 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57086c7a-9ec7-419e-8f56-060d1036996c admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:37.440628 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57086c7a-9ec7-419e-8f56-060d1036996c admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:37.442888 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57086c7a-9ec7-419e-8f56-060d1036996c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:37.448560 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57086c7a-9ec7-419e-8f56-060d1036996c admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:37.467179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 924/1847] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:37 2022] GET /identity/v3/roles => generated 1427 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:37.468974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8c780-842e-4b65-bdeb-825565b1b965 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:37.470118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8c780-842e-4b65-bdeb-825565b1b965 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:37.470633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8c780-842e-4b65-bdeb-825565b1b965 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:37.472748 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 925/1848] 10.222.0.100 () {54 vars in 926 bytes} [Tue Aug 2 09:11:37 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:11:37.473797 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8c780-842e-4b65-bdeb-825565b1b965 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:37.493571 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8c780-842e-4b65-bdeb-825565b1b965 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:37.503764 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83dca6-c29a-4e8f-aa37-7cc553d1e826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:37.505591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83dca6-c29a-4e8f-aa37-7cc553d1e826 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:37.506092 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83dca6-c29a-4e8f-aa37-7cc553d1e826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:37.506575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83dca6-c29a-4e8f-aa37-7cc553d1e826 admin admin] PATH_INFO: `/v3/projects/e908e08a518642b1a69a0fddc08441e8/users/43aee6558d614ad58770deea1e607b0f/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:37.508756 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:37.510107 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb83dca6-c29a-4e8f-aa37-7cc553d1e826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e908e08a518642b1a69a0fddc08441e8, user_id=43aee6558d614ad58770deea1e607b0f, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:37.516519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb83dca6-c29a-4e8f-aa37-7cc553d1e826 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:37.547432 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 926/1849] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:37 2022] PUT /identity/v3/projects/e908e08a518642b1a69a0fddc08441e8/users/43aee6558d614ad58770deea1e607b0f/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:37.559650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4bcd4-013c-4646-85b3-4ca25bb7f9ec None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:37.559957 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4bcd4-013c-4646-85b3-4ca25bb7f9ec None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:37.560225 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4bcd4-013c-4646-85b3-4ca25bb7f9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:37.568163 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2bbb2c-30c4-4790-af57-19f5fdb8f265 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:37.572868 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2bbb2c-30c4-4790-af57-19f5fdb8f265 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:37.592475 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 924/1850] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:37 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:37.603285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-f3e4bcd4-013c-4646-85b3-4ca25bb7f9ec None None] MFA Rules not processed for user `43aee6558d614ad58770deea1e607b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:37.604920 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f3e4bcd4-013c-4646-85b3-4ca25bb7f9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:37.633376 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e25a44-e24f-40ff-b2a9-302ece290b51 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:37.637470 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e25a44-e24f-40ff-b2a9-302ece290b51 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:37.637470 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e25a44-e24f-40ff-b2a9-302ece290b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:37.676846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a1e25a44-e24f-40ff-b2a9-302ece290b51 None None] MFA Rules not processed for user `49a0c1a4a6cf43e28ec1b8488c681848`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:37.678385 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a1e25a44-e24f-40ff-b2a9-302ece290b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:37.737640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 927/1851] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:37 2022] POST /identity/v3/auth/tokens => generated 3355 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:11:37.739459 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4bcd4-013c-4646-85b3-4ca25bb7f9ec None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:37.839620 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4bcd4-013c-4646-85b3-4ca25bb7f9ec None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:37.843572 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20ffce-e357-4c23-a225-b5749780fd3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:37.844355 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20ffce-e357-4c23-a225-b5749780fd3f service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:37.844758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20ffce-e357-4c23-a225-b5749780fd3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:37.845140 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20ffce-e357-4c23-a225-b5749780fd3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:37.848576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 925/1852] 10.222.0.100 () {58 vars in 1023 bytes} [Tue Aug 2 09:11:37 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:37.849875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e25a44-e24f-40ff-b2a9-302ece290b51 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:37.863512 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e25a44-e24f-40ff-b2a9-302ece290b51 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:37.867826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc880c-cbca-41ac-bb4e-ed836780fe99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:37.869155 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc880c-cbca-41ac-bb4e-ed836780fe99 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:37.869781 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc880c-cbca-41ac-bb4e-ed836780fe99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:37.870125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc880c-cbca-41ac-bb4e-ed836780fe99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:37.884701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc880c-cbca-41ac-bb4e-ed836780fe99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:37.886730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc880c-cbca-41ac-bb4e-ed836780fe99 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:37.905261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 926/1853] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:37 2022] GET /identity/v3/auth/tokens => generated 3355 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:37.936274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20ffce-e357-4c23-a225-b5749780fd3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:37.940498 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20ffce-e357-4c23-a225-b5749780fd3f service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:37.951003 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc880c-cbca-41ac-bb4e-ed836780fe99 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:37.952908 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-30cc880c-cbca-41ac-bb4e-ed836780fe99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:37.957531 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 928/1854] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:37.992592 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc880c-cbca-41ac-bb4e-ed836780fe99 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:38.010701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb85a5b-b2b4-443f-9177-414a76d217df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:38.011345 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb85a5b-b2b4-443f-9177-414a76d217df service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:38.011635 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb85a5b-b2b4-443f-9177-414a76d217df service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:38.011913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb85a5b-b2b4-443f-9177-414a76d217df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:38.053062 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb85a5b-b2b4-443f-9177-414a76d217df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:38.054990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb85a5b-b2b4-443f-9177-414a76d217df service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:38.080081 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 927/1855] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:37 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:11:38.428873 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20ffce-e357-4c23-a225-b5749780fd3f service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:38.440618 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20ffce-e357-4c23-a225-b5749780fd3f service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:38.451836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1c667-91a6-4a0c-82cc-7ee377378e7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:38.452475 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1c667-91a6-4a0c-82cc-7ee377378e7f service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:38.452756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1c667-91a6-4a0c-82cc-7ee377378e7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:38.453186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1c667-91a6-4a0c-82cc-7ee377378e7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:38.474976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:38.606219 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1c667-91a6-4a0c-82cc-7ee377378e7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:38.609307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1c667-91a6-4a0c-82cc-7ee377378e7f service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:38.649222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 929/1856] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:38 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:41.051765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb85a5b-b2b4-443f-9177-414a76d217df service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:41.162782 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb85a5b-b2b4-443f-9177-414a76d217df service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:41.167262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837f6ae-eb1c-4fd1-9742-8138ee0ea9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:41.167924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837f6ae-eb1c-4fd1-9742-8138ee0ea9f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:41.168207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837f6ae-eb1c-4fd1-9742-8138ee0ea9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:41.168562 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837f6ae-eb1c-4fd1-9742-8138ee0ea9f5 admin admin] PATH_INFO: `/v3/users/a538edc2994744de8b12549a8df4892d` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:41.170401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837f6ae-eb1c-4fd1-9742-8138ee0ea9f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a538edc2994744de8b12549a8df4892d)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:41.171667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837f6ae-eb1c-4fd1-9742-8138ee0ea9f5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:41.237599 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9837f6ae-eb1c-4fd1-9742-8138ee0ea9f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a538edc2994744de8b12549a8df4892d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:41.238951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:41.258156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9837f6ae-eb1c-4fd1-9742-8138ee0ea9f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a538edc2994744de8b12549a8df4892d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:41.258942 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9837f6ae-eb1c-4fd1-9742-8138ee0ea9f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a538edc2994744de8b12549a8df4892d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:41.273146 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9837f6ae-eb1c-4fd1-9742-8138ee0ea9f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a538edc2994744de8b12549a8df4892d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:41.273886 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9837f6ae-eb1c-4fd1-9742-8138ee0ea9f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a538edc2994744de8b12549a8df4892d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:41.307958 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 928/1857] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:11:41 2022] DELETE /identity/v3/users/a538edc2994744de8b12549a8df4892d => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:11:41.314121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a33a77-e99d-4068-9e73-a2b9fa0b6fef None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:41.314627 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a33a77-e99d-4068-9e73-a2b9fa0b6fef None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:41.318348 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a33a77-e99d-4068-9e73-a2b9fa0b6fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:41.349374 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-d9a33a77-e99d-4068-9e73-a2b9fa0b6fef None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:41.350614 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d9a33a77-e99d-4068-9e73-a2b9fa0b6fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:41.416634 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 930/1858] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:41.427193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837f6ae-eb1c-4fd1-9742-8138ee0ea9f5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:41.428649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9837f6ae-eb1c-4fd1-9742-8138ee0ea9f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:41.583388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837f6ae-eb1c-4fd1-9742-8138ee0ea9f5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:41.587202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c6b94-cc5d-4393-a27c-afccb3b394dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:41.587877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c6b94-cc5d-4393-a27c-afccb3b394dd service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:41.588163 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c6b94-cc5d-4393-a27c-afccb3b394dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:41.588441 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c6b94-cc5d-4393-a27c-afccb3b394dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:41.603489 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c6b94-cc5d-4393-a27c-afccb3b394dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:41.604937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c6b94-cc5d-4393-a27c-afccb3b394dd service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:41.616179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 929/1859] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:41 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:42.166328 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a33a77-e99d-4068-9e73-a2b9fa0b6fef None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:42.168418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d9a33a77-e99d-4068-9e73-a2b9fa0b6fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:42.213211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a33a77-e99d-4068-9e73-a2b9fa0b6fef None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:42.218191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc1c29-dce1-4482-8207-c2ed060a0ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:42.219653 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc1c29-dce1-4482-8207-c2ed060a0ee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:42.219954 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc1c29-dce1-4482-8207-c2ed060a0ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:42.220230 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc1c29-dce1-4482-8207-c2ed060a0ee0 admin admin] PATH_INFO: `/v3/projects/9b165a73dfc44daeaacb2450d21db5e6` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:42.222197 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc1c29-dce1-4482-8207-c2ed060a0ee0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b165a73dfc44daeaacb2450d21db5e6)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:42.224229 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc1c29-dce1-4482-8207-c2ed060a0ee0 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:42.282905 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5ddc1c29-dce1-4482-8207-c2ed060a0ee0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b165a73dfc44daeaacb2450d21db5e6', 'request_id': 'req-5ddc1c29-dce1-4482-8207-c2ed060a0ee0'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:42.291575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5ddc1c29-dce1-4482-8207-c2ed060a0ee0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b165a73dfc44daeaacb2450d21db5e6', 'request_id': 'req-5ddc1c29-dce1-4482-8207-c2ed060a0ee0'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:42.302643 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5ddc1c29-dce1-4482-8207-c2ed060a0ee0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b165a73dfc44daeaacb2450d21db5e6', 'request_id': 'req-5ddc1c29-dce1-4482-8207-c2ed060a0ee0'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:42.319950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5ddc1c29-dce1-4482-8207-c2ed060a0ee0 admin admin] The token cache is being invalidate because project 9b165a73dfc44daeaacb2450d21db5e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:42.320886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5ddc1c29-dce1-4482-8207-c2ed060a0ee0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:42.321816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5ddc1c29-dce1-4482-8207-c2ed060a0ee0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:42.329169 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 931/1860] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:11:42 2022] DELETE /identity/v3/projects/9b165a73dfc44daeaacb2450d21db5e6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:42.330116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c6b94-cc5d-4393-a27c-afccb3b394dd service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:42.343646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c6b94-cc5d-4393-a27c-afccb3b394dd service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:42.489902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bab7b7-6177-4b3a-862e-d27950f6f5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:42.491823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bab7b7-6177-4b3a-862e-d27950f6f5f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:42.492288 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bab7b7-6177-4b3a-862e-d27950f6f5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:42.492716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bab7b7-6177-4b3a-862e-d27950f6f5f3 admin admin] PATH_INFO: `/v3/users/a32c3e035cf94e08a54e25c2bae31a6f` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:42.494308 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bab7b7-6177-4b3a-862e-d27950f6f5f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a32c3e035cf94e08a54e25c2bae31a6f)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:42.497033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bab7b7-6177-4b3a-862e-d27950f6f5f3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:42.653803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-60bab7b7-6177-4b3a-862e-d27950f6f5f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a32c3e035cf94e08a54e25c2bae31a6f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:42.672902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-60bab7b7-6177-4b3a-862e-d27950f6f5f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a32c3e035cf94e08a54e25c2bae31a6f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:42.674888 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-60bab7b7-6177-4b3a-862e-d27950f6f5f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a32c3e035cf94e08a54e25c2bae31a6f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:42.686752 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-60bab7b7-6177-4b3a-862e-d27950f6f5f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a32c3e035cf94e08a54e25c2bae31a6f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:42.687860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-60bab7b7-6177-4b3a-862e-d27950f6f5f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a32c3e035cf94e08a54e25c2bae31a6f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:42.711766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:42.725019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 930/1861] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:11:42 2022] DELETE /identity/v3/users/a32c3e035cf94e08a54e25c2bae31a6f => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:43.228956 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc1c29-dce1-4482-8207-c2ed060a0ee0 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:43.230581 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-5ddc1c29-dce1-4482-8207-c2ed060a0ee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:43.246085 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc1c29-dce1-4482-8207-c2ed060a0ee0 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:43.287687 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:43.320133 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a34d0e-b423-4d20-91a3-c650f0ee4642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:43.320793 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a34d0e-b423-4d20-91a3-c650f0ee4642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:43.320981 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a34d0e-b423-4d20-91a3-c650f0ee4642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:43.321296 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a34d0e-b423-4d20-91a3-c650f0ee4642 admin admin] PATH_INFO: `/v3/projects/2e0c8030442e4afa926cf27face695d1` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:43.322470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a34d0e-b423-4d20-91a3-c650f0ee4642 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e0c8030442e4afa926cf27face695d1)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:43.324381 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a34d0e-b423-4d20-91a3-c650f0ee4642 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:43.375068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-48a34d0e-b423-4d20-91a3-c650f0ee4642 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e0c8030442e4afa926cf27face695d1', 'request_id': 'req-48a34d0e-b423-4d20-91a3-c650f0ee4642'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:43.383968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-48a34d0e-b423-4d20-91a3-c650f0ee4642 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e0c8030442e4afa926cf27face695d1', 'request_id': 'req-48a34d0e-b423-4d20-91a3-c650f0ee4642'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:43.391888 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-48a34d0e-b423-4d20-91a3-c650f0ee4642 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e0c8030442e4afa926cf27face695d1', 'request_id': 'req-48a34d0e-b423-4d20-91a3-c650f0ee4642'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:43.404648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-48a34d0e-b423-4d20-91a3-c650f0ee4642 admin admin] The token cache is being invalidate because project 2e0c8030442e4afa926cf27face695d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:43.404948 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-48a34d0e-b423-4d20-91a3-c650f0ee4642 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:43.405671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-48a34d0e-b423-4d20-91a3-c650f0ee4642 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:43.407222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 932/1862] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:11:43 2022] DELETE /identity/v3/projects/2e0c8030442e4afa926cf27face695d1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:43.414211 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6441e-c22e-49c5-b45c-9e16dc9d5d90 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:43.414749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6441e-c22e-49c5-b45c-9e16dc9d5d90 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:43.414749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6441e-c22e-49c5-b45c-9e16dc9d5d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:43.441292 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-82f6441e-c22e-49c5-b45c-9e16dc9d5d90 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:43.442532 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-82f6441e-c22e-49c5-b45c-9e16dc9d5d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:43.526285 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 931/1863] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:43.534498 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68d80b-8d1b-422c-968a-431d09568740 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:43.534902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68d80b-8d1b-422c-968a-431d09568740 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:43.535158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68d80b-8d1b-422c-968a-431d09568740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:43.556294 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-ba68d80b-8d1b-422c-968a-431d09568740 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:43.557604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ba68d80b-8d1b-422c-968a-431d09568740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:43.566844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 933/1864] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:43.574092 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6441e-c22e-49c5-b45c-9e16dc9d5d90 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:43.584280 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6441e-c22e-49c5-b45c-9e16dc9d5d90 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:43.589223 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a1c5c-0c42-41a9-ae55-6216112ad54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:43.589858 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a1c5c-0c42-41a9-ae55-6216112ad54a admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:43.590144 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a1c5c-0c42-41a9-ae55-6216112ad54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:43.590420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a1c5c-0c42-41a9-ae55-6216112ad54a admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:43.591124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a1c5c-0c42-41a9-ae55-6216112ad54a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:11:43.591431 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a1c5c-0c42-41a9-ae55-6216112ad54a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:43.593056 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a1c5c-0c42-41a9-ae55-6216112ad54a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:43.605959 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 932/1865] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:11:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:11:43.610687 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68d80b-8d1b-422c-968a-431d09568740 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:43.622078 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68d80b-8d1b-422c-968a-431d09568740 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:43.628898 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789f4b9-1ef0-49af-af3a-1868e76309f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:43.630641 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789f4b9-1ef0-49af-af3a-1868e76309f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:43.631194 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789f4b9-1ef0-49af-af3a-1868e76309f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:43.632128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789f4b9-1ef0-49af-af3a-1868e76309f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:43.633605 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789f4b9-1ef0-49af-af3a-1868e76309f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:43.636370 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789f4b9-1ef0-49af-af3a-1868e76309f9 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:43.673490 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 934/1866] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:11:43 2022] POST /identity/v3/projects => generated 378 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:43.675710 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a1c5c-0c42-41a9-ae55-6216112ad54a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:43.690432 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a1c5c-0c42-41a9-ae55-6216112ad54a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:43.781313 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9e680-941f-4a10-92d9-8200e1f476c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:43.782111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9e680-941f-4a10-92d9-8200e1f476c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:43.782392 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9e680-941f-4a10-92d9-8200e1f476c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:43.782667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9e680-941f-4a10-92d9-8200e1f476c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:43.783851 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9e680-941f-4a10-92d9-8200e1f476c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:43.786477 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9e680-941f-4a10-92d9-8200e1f476c3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:43.828413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 933/1867] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:11:43 2022] POST /identity/v3/users => generated 290 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:43.829654 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789f4b9-1ef0-49af-af3a-1868e76309f9 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:43.840587 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789f4b9-1ef0-49af-af3a-1868e76309f9 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:43.848449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970083d-bcc1-426f-9fa8-0f6a5e63814f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:43.848449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970083d-bcc1-426f-9fa8-0f6a5e63814f admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:43.848749 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970083d-bcc1-426f-9fa8-0f6a5e63814f admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:43.848869 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970083d-bcc1-426f-9fa8-0f6a5e63814f admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:43.849565 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970083d-bcc1-426f-9fa8-0f6a5e63814f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:43.851755 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970083d-bcc1-426f-9fa8-0f6a5e63814f admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:43.864995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 935/1868] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:43 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:43.868909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9e680-941f-4a10-92d9-8200e1f476c3 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:43.882496 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9e680-941f-4a10-92d9-8200e1f476c3 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:43.887260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a090d-e50b-4c74-b108-23a60bc610b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:43.887894 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a090d-e50b-4c74-b108-23a60bc610b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:43.888180 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a090d-e50b-4c74-b108-23a60bc610b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:43.888456 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a090d-e50b-4c74-b108-23a60bc610b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:43.889190 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a090d-e50b-4c74-b108-23a60bc610b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:43.890830 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a090d-e50b-4c74-b108-23a60bc610b1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:43.902985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 934/1869] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:43 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:43.907237 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970083d-bcc1-426f-9fa8-0f6a5e63814f admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:43.919035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970083d-bcc1-426f-9fa8-0f6a5e63814f admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:43.924359 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b61917-6e6e-4baa-95ae-25a34fa9eb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:43.925953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b61917-6e6e-4baa-95ae-25a34fa9eb3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:43.926244 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b61917-6e6e-4baa-95ae-25a34fa9eb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:43.926549 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b61917-6e6e-4baa-95ae-25a34fa9eb3f admin admin] PATH_INFO: `/v3/projects/a1cc8acac44d4625b074d2793d97c120/users/2e1b302883cf4786b0369f911bb28228/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:43.950644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b61917-6e6e-4baa-95ae-25a34fa9eb3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1cc8acac44d4625b074d2793d97c120, user_id=2e1b302883cf4786b0369f911bb28228, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:43.953607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b61917-6e6e-4baa-95ae-25a34fa9eb3f admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:43.968701 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 936/1870] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:43 2022] PUT /identity/v3/projects/a1cc8acac44d4625b074d2793d97c120/users/2e1b302883cf4786b0369f911bb28228/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:43.980249 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5ad51-aace-4c5f-b206-9a2efcce2ddf None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:43.980536 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5ad51-aace-4c5f-b206-9a2efcce2ddf None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:43.980818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5ad51-aace-4c5f-b206-9a2efcce2ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:44.011047 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-72a5ad51-aace-4c5f-b206-9a2efcce2ddf None None] MFA Rules not processed for user `2e1b302883cf4786b0369f911bb28228`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:44.012259 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-72a5ad51-aace-4c5f-b206-9a2efcce2ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:44.056976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:44.084372 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 935/1871] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:11:44.095033 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b61917-6e6e-4baa-95ae-25a34fa9eb3f admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:44.097599 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e9b61917-6e6e-4baa-95ae-25a34fa9eb3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:44.184041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:44.198558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b61917-6e6e-4baa-95ae-25a34fa9eb3f admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:44.206450 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7d9a0-f321-44a2-984c-16d02e761668 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd959ddd087724547a11c3d4c796d8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:44.207503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7d9a0-f321-44a2-984c-16d02e761668 service glance] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:44.207840 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7d9a0-f321-44a2-984c-16d02e761668 service glance] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:44.208125 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7d9a0-f321-44a2-984c-16d02e761668 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:44.217454 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7d9a0-f321-44a2-984c-16d02e761668 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:44.218918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7d9a0-f321-44a2-984c-16d02e761668 service glance] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:44.229981 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 937/1872] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:44 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:45.218002 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d1204-24a4-4557-87db-4af8bddb1f53 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:45.219081 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d1204-24a4-4557-87db-4af8bddb1f53 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:45.219081 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d1204-24a4-4557-87db-4af8bddb1f53 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:45.220784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 936/1873] 10.222.0.100 () {54 vars in 926 bytes} [Tue Aug 2 09:11:45 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:11:45.230657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663738f-86dd-4af6-8adf-4fb9dda5960a None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:45.231207 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663738f-86dd-4af6-8adf-4fb9dda5960a None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:45.231207 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663738f-86dd-4af6-8adf-4fb9dda5960a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:45.256933 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-e663738f-86dd-4af6-8adf-4fb9dda5960a None None] MFA Rules not processed for user `49a0c1a4a6cf43e28ec1b8488c681848`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:45.258413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e663738f-86dd-4af6-8adf-4fb9dda5960a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:45.351780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 938/1874] 10.222.0.100 () {58 vars in 1023 bytes} [Tue Aug 2 09:11:45 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:45.459563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d1204-24a4-4557-87db-4af8bddb1f53 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:45.461497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ae6d1204-24a4-4557-87db-4af8bddb1f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:45.480349 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d1204-24a4-4557-87db-4af8bddb1f53 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:45.486396 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ae874-8ba3-4e53-8f52-677a0af3e1e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:45.487858 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ae874-8ba3-4e53-8f52-677a0af3e1e6 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:45.488253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ae874-8ba3-4e53-8f52-677a0af3e1e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:45.488654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ae874-8ba3-4e53-8f52-677a0af3e1e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:45.499245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ae874-8ba3-4e53-8f52-677a0af3e1e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:45.500830 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ae874-8ba3-4e53-8f52-677a0af3e1e6 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:45.512283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 937/1875] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:45 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:52.950262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663738f-86dd-4af6-8adf-4fb9dda5960a None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:52.951763 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e663738f-86dd-4af6-8adf-4fb9dda5960a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:53.035580 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663738f-86dd-4af6-8adf-4fb9dda5960a None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:53.040185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77105bc5-e882-4f07-a422-f09dec4487f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:53.040847 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77105bc5-e882-4f07-a422-f09dec4487f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:53.041158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77105bc5-e882-4f07-a422-f09dec4487f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:53.041445 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77105bc5-e882-4f07-a422-f09dec4487f4 admin admin] PATH_INFO: `/v3/users/194bd5ee756f42eba846bee2d1692bfb` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:53.042828 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77105bc5-e882-4f07-a422-f09dec4487f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=194bd5ee756f42eba846bee2d1692bfb)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:53.046684 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77105bc5-e882-4f07-a422-f09dec4487f4 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:53.107807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-77105bc5-e882-4f07-a422-f09dec4487f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '194bd5ee756f42eba846bee2d1692bfb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:53.108464 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-77105bc5-e882-4f07-a422-f09dec4487f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '194bd5ee756f42eba846bee2d1692bfb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:53.115859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-77105bc5-e882-4f07-a422-f09dec4487f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '194bd5ee756f42eba846bee2d1692bfb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:53.127628 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-77105bc5-e882-4f07-a422-f09dec4487f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '194bd5ee756f42eba846bee2d1692bfb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:53.132970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:53.134982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-77105bc5-e882-4f07-a422-f09dec4487f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '194bd5ee756f42eba846bee2d1692bfb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:53.142034 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 939/1876] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:11:52 2022] DELETE /identity/v3/users/194bd5ee756f42eba846bee2d1692bfb => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:53.147375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb9cbf-5e82-4769-85d3-6cf2cdf09bda None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:53.147939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb9cbf-5e82-4769-85d3-6cf2cdf09bda None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:53.148192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb9cbf-5e82-4769-85d3-6cf2cdf09bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:53.170602 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d3eb9cbf-5e82-4769-85d3-6cf2cdf09bda None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:53.171793 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d3eb9cbf-5e82-4769-85d3-6cf2cdf09bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:53.245335 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 938/1877] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:53.255881 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77105bc5-e882-4f07-a422-f09dec4487f4 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:53.257462 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-77105bc5-e882-4f07-a422-f09dec4487f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:53.340389 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77105bc5-e882-4f07-a422-f09dec4487f4 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:53.344298 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61628cb2-bfff-419a-8d24-5a58261faa45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:53.345372 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61628cb2-bfff-419a-8d24-5a58261faa45 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:53.345848 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61628cb2-bfff-419a-8d24-5a58261faa45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:53.346228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61628cb2-bfff-419a-8d24-5a58261faa45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:53.359192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61628cb2-bfff-419a-8d24-5a58261faa45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:53.360566 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61628cb2-bfff-419a-8d24-5a58261faa45 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:53.375214 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 940/1878] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:53 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:53.895135 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb9cbf-5e82-4769-85d3-6cf2cdf09bda None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:53.902910 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d3eb9cbf-5e82-4769-85d3-6cf2cdf09bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:53.940191 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb9cbf-5e82-4769-85d3-6cf2cdf09bda None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:53.960748 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ac6a1-1023-4389-b5d0-91107ae7d05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:53.961705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ac6a1-1023-4389-b5d0-91107ae7d05d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:53.962138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ac6a1-1023-4389-b5d0-91107ae7d05d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:53.962605 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ac6a1-1023-4389-b5d0-91107ae7d05d admin admin] PATH_INFO: `/v3/projects/34f66feb02ae4e88bb9cbb96fca6f734` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:53.964003 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ac6a1-1023-4389-b5d0-91107ae7d05d admin admin] RBAC: Authorizing `identity:delete_project(project_id=34f66feb02ae4e88bb9cbb96fca6f734)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:53.966439 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ac6a1-1023-4389-b5d0-91107ae7d05d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:54.056231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fa1ac6a1-1023-4389-b5d0-91107ae7d05d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34f66feb02ae4e88bb9cbb96fca6f734', 'request_id': 'req-fa1ac6a1-1023-4389-b5d0-91107ae7d05d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:54.066985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fa1ac6a1-1023-4389-b5d0-91107ae7d05d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34f66feb02ae4e88bb9cbb96fca6f734', 'request_id': 'req-fa1ac6a1-1023-4389-b5d0-91107ae7d05d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:54.080753 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fa1ac6a1-1023-4389-b5d0-91107ae7d05d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34f66feb02ae4e88bb9cbb96fca6f734', 'request_id': 'req-fa1ac6a1-1023-4389-b5d0-91107ae7d05d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:54.097138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fa1ac6a1-1023-4389-b5d0-91107ae7d05d admin admin] The token cache is being invalidate because project 34f66feb02ae4e88bb9cbb96fca6f734 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:54.097611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fa1ac6a1-1023-4389-b5d0-91107ae7d05d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:54.098488 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fa1ac6a1-1023-4389-b5d0-91107ae7d05d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:54.100711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 939/1879] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:11:53 2022] DELETE /identity/v3/projects/34f66feb02ae4e88bb9cbb96fca6f734 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:54.105534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61628cb2-bfff-419a-8d24-5a58261faa45 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:54.162954 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61628cb2-bfff-419a-8d24-5a58261faa45 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:54.315944 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34edc9-0f12-4421-8502-f79a85ba82c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:54.316669 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34edc9-0f12-4421-8502-f79a85ba82c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:54.316965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34edc9-0f12-4421-8502-f79a85ba82c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:54.317291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34edc9-0f12-4421-8502-f79a85ba82c7 admin admin] PATH_INFO: `/v3/users/43aee6558d614ad58770deea1e607b0f` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:54.318487 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34edc9-0f12-4421-8502-f79a85ba82c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43aee6558d614ad58770deea1e607b0f)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:54.322300 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34edc9-0f12-4421-8502-f79a85ba82c7 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:54.382365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-1f34edc9-0f12-4421-8502-f79a85ba82c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43aee6558d614ad58770deea1e607b0f'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:54.383098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-1f34edc9-0f12-4421-8502-f79a85ba82c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43aee6558d614ad58770deea1e607b0f'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:54.391163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-1f34edc9-0f12-4421-8502-f79a85ba82c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43aee6558d614ad58770deea1e607b0f'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:54.405507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-1f34edc9-0f12-4421-8502-f79a85ba82c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43aee6558d614ad58770deea1e607b0f'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:54.413963 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-1f34edc9-0f12-4421-8502-f79a85ba82c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43aee6558d614ad58770deea1e607b0f'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:54.420902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 941/1880] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:11:54 2022] DELETE /identity/v3/users/43aee6558d614ad58770deea1e607b0f => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:54.766554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ac6a1-1023-4389-b5d0-91107ae7d05d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:54.769051 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fa1ac6a1-1023-4389-b5d0-91107ae7d05d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:54.785434 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ac6a1-1023-4389-b5d0-91107ae7d05d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:54.837990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:54.849296 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd30871-72ed-4389-b4e0-5ee46faeb1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:54.849959 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd30871-72ed-4389-b4e0-5ee46faeb1dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:54.850245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd30871-72ed-4389-b4e0-5ee46faeb1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:54.850524 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd30871-72ed-4389-b4e0-5ee46faeb1dc admin admin] PATH_INFO: `/v3/projects/e908e08a518642b1a69a0fddc08441e8` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:54.851704 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd30871-72ed-4389-b4e0-5ee46faeb1dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e908e08a518642b1a69a0fddc08441e8)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:54.853644 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd30871-72ed-4389-b4e0-5ee46faeb1dc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:54.919567 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cfd30871-72ed-4389-b4e0-5ee46faeb1dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e908e08a518642b1a69a0fddc08441e8', 'request_id': 'req-cfd30871-72ed-4389-b4e0-5ee46faeb1dc'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:54.927415 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cfd30871-72ed-4389-b4e0-5ee46faeb1dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e908e08a518642b1a69a0fddc08441e8', 'request_id': 'req-cfd30871-72ed-4389-b4e0-5ee46faeb1dc'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:54.935724 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cfd30871-72ed-4389-b4e0-5ee46faeb1dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e908e08a518642b1a69a0fddc08441e8', 'request_id': 'req-cfd30871-72ed-4389-b4e0-5ee46faeb1dc'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:54.947604 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cfd30871-72ed-4389-b4e0-5ee46faeb1dc admin admin] The token cache is being invalidate because project e908e08a518642b1a69a0fddc08441e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:54.947852 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cfd30871-72ed-4389-b4e0-5ee46faeb1dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:54.948601 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cfd30871-72ed-4389-b4e0-5ee46faeb1dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:54.950384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 940/1881] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:11:54 2022] DELETE /identity/v3/projects/e908e08a518642b1a69a0fddc08441e8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:54.955906 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452f6a0-cfa4-4aac-9a0a-68f208f2b098 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:54.956120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452f6a0-cfa4-4aac-9a0a-68f208f2b098 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:54.956600 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452f6a0-cfa4-4aac-9a0a-68f208f2b098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:54.977980 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-d452f6a0-cfa4-4aac-9a0a-68f208f2b098 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:54.979332 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d452f6a0-cfa4-4aac-9a0a-68f208f2b098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:54.993448 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:55.048789 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 942/1882] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:55.059156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e2b52-0f11-44e8-a89b-f9789dbbc1f1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:55.059745 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e2b52-0f11-44e8-a89b-f9789dbbc1f1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:55.059745 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e2b52-0f11-44e8-a89b-f9789dbbc1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:55.084054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e95e2b52-0f11-44e8-a89b-f9789dbbc1f1 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:55.085381 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e95e2b52-0f11-44e8-a89b-f9789dbbc1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:55.096297 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 941/1883] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:11:55.101720 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452f6a0-cfa4-4aac-9a0a-68f208f2b098 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:55.112072 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452f6a0-cfa4-4aac-9a0a-68f208f2b098 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:55.117535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366dfbb-5f75-4023-a0d8-f0bbeaca5b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:55.118091 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366dfbb-5f75-4023-a0d8-f0bbeaca5b65 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:55.118376 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366dfbb-5f75-4023-a0d8-f0bbeaca5b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:55.118664 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366dfbb-5f75-4023-a0d8-f0bbeaca5b65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:55.119570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366dfbb-5f75-4023-a0d8-f0bbeaca5b65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:11:55.119919 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366dfbb-5f75-4023-a0d8-f0bbeaca5b65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:55.121522 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366dfbb-5f75-4023-a0d8-f0bbeaca5b65 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:55.134848 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 943/1884] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:11:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:11:55.139767 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e2b52-0f11-44e8-a89b-f9789dbbc1f1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:55.150281 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e2b52-0f11-44e8-a89b-f9789dbbc1f1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:55.155849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f0d87-c4ee-4183-ae8e-3931590199e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:55.156494 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f0d87-c4ee-4183-ae8e-3931590199e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:55.156779 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f0d87-c4ee-4183-ae8e-3931590199e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:55.157897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f0d87-c4ee-4183-ae8e-3931590199e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:55.158593 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f0d87-c4ee-4183-ae8e-3931590199e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:55.160420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f0d87-c4ee-4183-ae8e-3931590199e1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:55.203042 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 942/1885] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:11:55 2022] POST /identity/v3/projects => generated 384 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:55.208088 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366dfbb-5f75-4023-a0d8-f0bbeaca5b65 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:55.219138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366dfbb-5f75-4023-a0d8-f0bbeaca5b65 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:55.332490 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e986a47-8524-47f5-84d4-019e8f97985a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:55.333495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e986a47-8524-47f5-84d4-019e8f97985a admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:55.334012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e986a47-8524-47f5-84d4-019e8f97985a admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:55.334500 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e986a47-8524-47f5-84d4-019e8f97985a admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:55.349919 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e986a47-8524-47f5-84d4-019e8f97985a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:55.353901 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e986a47-8524-47f5-84d4-019e8f97985a admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:55.403572 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 944/1886] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:11:55 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:55.407386 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f0d87-c4ee-4183-ae8e-3931590199e1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:55.431490 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f0d87-c4ee-4183-ae8e-3931590199e1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:55.448789 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59e057-dbde-45ce-98f0-c5fed35ee46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:55.453597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59e057-dbde-45ce-98f0-c5fed35ee46d admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:55.456698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59e057-dbde-45ce-98f0-c5fed35ee46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:55.457619 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59e057-dbde-45ce-98f0-c5fed35ee46d admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:55.461324 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c59e057-dbde-45ce-98f0-c5fed35ee46d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:55.466714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c59e057-dbde-45ce-98f0-c5fed35ee46d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:55.514859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e986a47-8524-47f5-84d4-019e8f97985a admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:55.515063 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 943/1887] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:55 2022] GET /identity/v3/roles => generated 1427 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:55.530943 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e986a47-8524-47f5-84d4-019e8f97985a admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:55.554113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679912f-ec2b-4699-a29e-270b5a5af734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:55.554888 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679912f-ec2b-4699-a29e-270b5a5af734 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:55.555137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679912f-ec2b-4699-a29e-270b5a5af734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:55.555329 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679912f-ec2b-4699-a29e-270b5a5af734 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:55.556311 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679912f-ec2b-4699-a29e-270b5a5af734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:55.559691 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679912f-ec2b-4699-a29e-270b5a5af734 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:55.594234 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 945/1888] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:55 2022] GET /identity/v3/roles => generated 1427 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:11:55.609940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59e057-dbde-45ce-98f0-c5fed35ee46d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:55.628116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59e057-dbde-45ce-98f0-c5fed35ee46d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:55.637796 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214c836-210a-43b7-b126-d4d37070201b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:55.638874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214c836-210a-43b7-b126-d4d37070201b admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:55.639235 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214c836-210a-43b7-b126-d4d37070201b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:55.639434 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214c836-210a-43b7-b126-d4d37070201b admin admin] PATH_INFO: `/v3/projects/5ce3752acd054c48bda4e788ccab4b9c/users/f7fec536945549a7944d63ccc8978c94/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:55.681905 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214c836-210a-43b7-b126-d4d37070201b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ce3752acd054c48bda4e788ccab4b9c, user_id=f7fec536945549a7944d63ccc8978c94, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:55.694217 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214c836-210a-43b7-b126-d4d37070201b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:55.733710 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 944/1889] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:55 2022] PUT /identity/v3/projects/5ce3752acd054c48bda4e788ccab4b9c/users/f7fec536945549a7944d63ccc8978c94/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:55.733965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d89a0a-4a75-4c1c-ae74-8fcb34ebbfc3 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:55.734435 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d89a0a-4a75-4c1c-ae74-8fcb34ebbfc3 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:55.734918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d89a0a-4a75-4c1c-ae74-8fcb34ebbfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:55.843397 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-c6d89a0a-4a75-4c1c-ae74-8fcb34ebbfc3 None None] MFA Rules not processed for user `f7fec536945549a7944d63ccc8978c94`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:55.844726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c6d89a0a-4a75-4c1c-ae74-8fcb34ebbfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:56.061796 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 946/1890] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:56.068580 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214c836-210a-43b7-b126-d4d37070201b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:56.179540 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214c836-210a-43b7-b126-d4d37070201b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:56.184423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56b72f-850f-40b4-bea1-2b81f0ddb19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:56.185058 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56b72f-850f-40b4-bea1-2b81f0ddb19f admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:56.185745 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56b72f-850f-40b4-bea1-2b81f0ddb19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:56.185745 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56b72f-850f-40b4-bea1-2b81f0ddb19f admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:56.186345 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e56b72f-850f-40b4-bea1-2b81f0ddb19f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:56.188339 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e56b72f-850f-40b4-bea1-2b81f0ddb19f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:56.226960 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 945/1891] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:11:56 2022] POST /identity/v3/projects => generated 384 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:56.227620 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d89a0a-4a75-4c1c-ae74-8fcb34ebbfc3 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:56.239942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d89a0a-4a75-4c1c-ae74-8fcb34ebbfc3 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:56.249427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:56.317433 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec26f05-d3ee-46a6-9ee1-ac057b59aa2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:56.318063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec26f05-d3ee-46a6-9ee1-ac057b59aa2f admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:56.318352 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec26f05-d3ee-46a6-9ee1-ac057b59aa2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:56.318635 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec26f05-d3ee-46a6-9ee1-ac057b59aa2f admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:56.319305 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec26f05-d3ee-46a6-9ee1-ac057b59aa2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:56.321270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec26f05-d3ee-46a6-9ee1-ac057b59aa2f admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:56.342168 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 947/1892] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:11:56 2022] POST /identity/v3/users => generated 299 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:56.346287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56b72f-850f-40b4-bea1-2b81f0ddb19f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:56.352836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:56.357276 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56b72f-850f-40b4-bea1-2b81f0ddb19f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:56.364141 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85ea2e-d9d4-4f9f-a650-9f24c8408f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:56.364915 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85ea2e-d9d4-4f9f-a650-9f24c8408f08 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:56.365354 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85ea2e-d9d4-4f9f-a650-9f24c8408f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:56.365790 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85ea2e-d9d4-4f9f-a650-9f24c8408f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:56.366486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85ea2e-d9d4-4f9f-a650-9f24c8408f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:56.368071 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85ea2e-d9d4-4f9f-a650-9f24c8408f08 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:56.379346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 946/1893] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:56 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:56.383767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec26f05-d3ee-46a6-9ee1-ac057b59aa2f admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:56.393920 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec26f05-d3ee-46a6-9ee1-ac057b59aa2f admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:56.398723 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049cea76-0831-45d6-9f18-0a0827216d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:56.399394 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cea76-0831-45d6-9f18-0a0827216d47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:56.399758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cea76-0831-45d6-9f18-0a0827216d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:56.400039 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cea76-0831-45d6-9f18-0a0827216d47 admin admin] PATH_INFO: `/v3/projects/9c68b180811c4fcfb8873ca0aff606fb/users/bccda132fd3c49a392c14b45147fdf63/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:56.415473 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049cea76-0831-45d6-9f18-0a0827216d47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c68b180811c4fcfb8873ca0aff606fb, user_id=bccda132fd3c49a392c14b45147fdf63, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:56.418217 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049cea76-0831-45d6-9f18-0a0827216d47 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:56.433737 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 948/1894] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:56 2022] PUT /identity/v3/projects/9c68b180811c4fcfb8873ca0aff606fb/users/bccda132fd3c49a392c14b45147fdf63/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:56.438066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85ea2e-d9d4-4f9f-a650-9f24c8408f08 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:56.453645 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85ea2e-d9d4-4f9f-a650-9f24c8408f08 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:56.518740 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d343ffe-ac75-4c42-9ecc-5d84ca77433f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:56.519611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d343ffe-ac75-4c42-9ecc-5d84ca77433f admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:56.520004 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d343ffe-ac75-4c42-9ecc-5d84ca77433f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:56.520404 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d343ffe-ac75-4c42-9ecc-5d84ca77433f admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:56.521144 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d343ffe-ac75-4c42-9ecc-5d84ca77433f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:56.522757 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d343ffe-ac75-4c42-9ecc-5d84ca77433f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:56.534102 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 947/1895] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:11:56 2022] GET /identity/v3/roles => generated 1427 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:11:56.539145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049cea76-0831-45d6-9f18-0a0827216d47 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:56.550719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049cea76-0831-45d6-9f18-0a0827216d47 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:56.555509 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae1645f-b46e-48a9-9a76-8937320e0bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:56.556188 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1645f-b46e-48a9-9a76-8937320e0bd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:56.556464 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1645f-b46e-48a9-9a76-8937320e0bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:56.556753 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1645f-b46e-48a9-9a76-8937320e0bd3 admin admin] PATH_INFO: `/v3/projects/9c68b180811c4fcfb8873ca0aff606fb/users/bccda132fd3c49a392c14b45147fdf63/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:56.558833 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae1645f-b46e-48a9-9a76-8937320e0bd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c68b180811c4fcfb8873ca0aff606fb, user_id=bccda132fd3c49a392c14b45147fdf63, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:56.561535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae1645f-b46e-48a9-9a76-8937320e0bd3 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:56.573497 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 949/1896] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:11:56 2022] PUT /identity/v3/projects/9c68b180811c4fcfb8873ca0aff606fb/users/bccda132fd3c49a392c14b45147fdf63/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:56.586519 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46fb04-9b0c-4551-a234-04765a76775a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:56.586813 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46fb04-9b0c-4551-a234-04765a76775a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:56.587106 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46fb04-9b0c-4551-a234-04765a76775a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:56.618906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1f46fb04-9b0c-4551-a234-04765a76775a None None] MFA Rules not processed for user `bccda132fd3c49a392c14b45147fdf63`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:56.620147 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1f46fb04-9b0c-4551-a234-04765a76775a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:56.692745 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 948/1897] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:11:56 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:11:56.703510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae1645f-b46e-48a9-9a76-8937320e0bd3 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:56.705255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3ae1645f-b46e-48a9-9a76-8937320e0bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:56.764109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae1645f-b46e-48a9-9a76-8937320e0bd3 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:56.767805 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0a0f1-2f24-4feb-a095-aae95c9fcad6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:56.768433 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0a0f1-2f24-4feb-a095-aae95c9fcad6 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:56.768713 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0a0f1-2f24-4feb-a095-aae95c9fcad6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:56.768987 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0a0f1-2f24-4feb-a095-aae95c9fcad6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:56.778793 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0a0f1-2f24-4feb-a095-aae95c9fcad6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:56.780478 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0a0f1-2f24-4feb-a095-aae95c9fcad6 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:56.792934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 950/1898] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:56 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:56.912415 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46fb04-9b0c-4551-a234-04765a76775a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:56.914284 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1f46fb04-9b0c-4551-a234-04765a76775a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:56.928697 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46fb04-9b0c-4551-a234-04765a76775a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:56.932786 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ccbb5-4ffc-4382-b99f-02a4241caa7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:56.933497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ccbb5-4ffc-4382-b99f-02a4241caa7c service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:56.933782 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ccbb5-4ffc-4382-b99f-02a4241caa7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:56.934063 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ccbb5-4ffc-4382-b99f-02a4241caa7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:57.011158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ccbb5-4ffc-4382-b99f-02a4241caa7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:57.012824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ccbb5-4ffc-4382-b99f-02a4241caa7c service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:57.026932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 949/1899] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:11:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:12:01.868766 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0a0f1-2f24-4feb-a095-aae95c9fcad6 service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:01.944616 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0a0f1-2f24-4feb-a095-aae95c9fcad6 service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:01.949812 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e1931-01c9-44ff-8c10-d98e18cf09aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:01.950472 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e1931-01c9-44ff-8c10-d98e18cf09aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:01.950753 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e1931-01c9-44ff-8c10-d98e18cf09aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:01.951060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e1931-01c9-44ff-8c10-d98e18cf09aa admin admin] PATH_INFO: `/v3/users/f7fec536945549a7944d63ccc8978c94` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:01.952245 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766e1931-01c9-44ff-8c10-d98e18cf09aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7fec536945549a7944d63ccc8978c94)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:01.954260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766e1931-01c9-44ff-8c10-d98e18cf09aa admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:01.966175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:02.018191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-766e1931-01c9-44ff-8c10-d98e18cf09aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7fec536945549a7944d63ccc8978c94'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:02.018853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-766e1931-01c9-44ff-8c10-d98e18cf09aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7fec536945549a7944d63ccc8978c94'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:02.026836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-766e1931-01c9-44ff-8c10-d98e18cf09aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7fec536945549a7944d63ccc8978c94'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:02.042674 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-766e1931-01c9-44ff-8c10-d98e18cf09aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7fec536945549a7944d63ccc8978c94'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:02.050558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-766e1931-01c9-44ff-8c10-d98e18cf09aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7fec536945549a7944d63ccc8978c94'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:02.057652 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 951/1900] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:12:01 2022] DELETE /identity/v3/users/f7fec536945549a7944d63ccc8978c94 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:02.062252 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36245e15-2645-4de6-861b-141992b306ee None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:02.062786 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36245e15-2645-4de6-861b-141992b306ee None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:02.062786 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36245e15-2645-4de6-861b-141992b306ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:02.083685 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-36245e15-2645-4de6-861b-141992b306ee None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:02.084932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-36245e15-2645-4de6-861b-141992b306ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:02.093710 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:02.166534 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 950/1901] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:02.176509 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e1931-01c9-44ff-8c10-d98e18cf09aa admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:02.177997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-766e1931-01c9-44ff-8c10-d98e18cf09aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:02.243101 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e1931-01c9-44ff-8c10-d98e18cf09aa admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:02.246774 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0116931-1fd7-4e7e-b4a6-4aa543a0d47f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:02.247404 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0116931-1fd7-4e7e-b4a6-4aa543a0d47f service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:02.247692 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0116931-1fd7-4e7e-b4a6-4aa543a0d47f service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:02.247975 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0116931-1fd7-4e7e-b4a6-4aa543a0d47f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:02.258220 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0116931-1fd7-4e7e-b4a6-4aa543a0d47f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:02.259695 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0116931-1fd7-4e7e-b4a6-4aa543a0d47f service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:02.270608 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 952/1902] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:02 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:12:02.564280 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36245e15-2645-4de6-861b-141992b306ee None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:02.566096 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-36245e15-2645-4de6-861b-141992b306ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:02.578788 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36245e15-2645-4de6-861b-141992b306ee None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:02.584366 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b3661-5555-43a7-9f9d-971918688a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:02.585081 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b3661-5555-43a7-9f9d-971918688a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:02.585406 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b3661-5555-43a7-9f9d-971918688a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:02.585696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b3661-5555-43a7-9f9d-971918688a5a admin admin] PATH_INFO: `/v3/projects/5ce3752acd054c48bda4e788ccab4b9c` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:02.586944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b3661-5555-43a7-9f9d-971918688a5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ce3752acd054c48bda4e788ccab4b9c)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:02.588905 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b3661-5555-43a7-9f9d-971918688a5a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:02.649076 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-aa3b3661-5555-43a7-9f9d-971918688a5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ce3752acd054c48bda4e788ccab4b9c', 'request_id': 'req-aa3b3661-5555-43a7-9f9d-971918688a5a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:02.656098 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-aa3b3661-5555-43a7-9f9d-971918688a5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ce3752acd054c48bda4e788ccab4b9c', 'request_id': 'req-aa3b3661-5555-43a7-9f9d-971918688a5a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:02.667364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-aa3b3661-5555-43a7-9f9d-971918688a5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ce3752acd054c48bda4e788ccab4b9c', 'request_id': 'req-aa3b3661-5555-43a7-9f9d-971918688a5a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:02.681833 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-aa3b3661-5555-43a7-9f9d-971918688a5a admin admin] The token cache is being invalidate because project 5ce3752acd054c48bda4e788ccab4b9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:12:02.682169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-aa3b3661-5555-43a7-9f9d-971918688a5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:02.682865 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-aa3b3661-5555-43a7-9f9d-971918688a5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:02.684795 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 951/1903] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:12:02 2022] DELETE /identity/v3/projects/5ce3752acd054c48bda4e788ccab4b9c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:02.689201 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0116931-1fd7-4e7e-b4a6-4aa543a0d47f service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:02.701805 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0116931-1fd7-4e7e-b4a6-4aa543a0d47f service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:02.787853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de829fb-f096-4a95-a746-b94a435ba2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:02.788729 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de829fb-f096-4a95-a746-b94a435ba2e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:02.789023 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de829fb-f096-4a95-a746-b94a435ba2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:02.789391 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de829fb-f096-4a95-a746-b94a435ba2e3 admin admin] PATH_INFO: `/v3/users/bccda132fd3c49a392c14b45147fdf63` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:02.790699 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de829fb-f096-4a95-a746-b94a435ba2e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bccda132fd3c49a392c14b45147fdf63)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:02.792797 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de829fb-f096-4a95-a746-b94a435ba2e3 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:02.839928 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:02.855314 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8de829fb-f096-4a95-a746-b94a435ba2e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bccda132fd3c49a392c14b45147fdf63'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:02.856142 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8de829fb-f096-4a95-a746-b94a435ba2e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bccda132fd3c49a392c14b45147fdf63'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:02.866049 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8de829fb-f096-4a95-a746-b94a435ba2e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bccda132fd3c49a392c14b45147fdf63'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:02.884351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8de829fb-f096-4a95-a746-b94a435ba2e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bccda132fd3c49a392c14b45147fdf63'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:02.893157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8de829fb-f096-4a95-a746-b94a435ba2e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bccda132fd3c49a392c14b45147fdf63'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:02.900424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 953/1904] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:12:02 2022] DELETE /identity/v3/users/bccda132fd3c49a392c14b45147fdf63 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:03.494097 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b3661-5555-43a7-9f9d-971918688a5a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:03.495613 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-aa3b3661-5555-43a7-9f9d-971918688a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:03.509827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b3661-5555-43a7-9f9d-971918688a5a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:03.573042 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf895e-62fe-4731-84e3-d1abca163a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:03.573730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf895e-62fe-4731-84e3-d1abca163a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:03.574024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf895e-62fe-4731-84e3-d1abca163a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:03.574311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf895e-62fe-4731-84e3-d1abca163a89 admin admin] PATH_INFO: `/v3/projects/9c68b180811c4fcfb8873ca0aff606fb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:03.575482 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf895e-62fe-4731-84e3-d1abca163a89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c68b180811c4fcfb8873ca0aff606fb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:03.577464 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf895e-62fe-4731-84e3-d1abca163a89 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:03.618890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:03.631188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b6cf895e-62fe-4731-84e3-d1abca163a89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c68b180811c4fcfb8873ca0aff606fb', 'request_id': 'req-b6cf895e-62fe-4731-84e3-d1abca163a89'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:03.637780 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b6cf895e-62fe-4731-84e3-d1abca163a89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c68b180811c4fcfb8873ca0aff606fb', 'request_id': 'req-b6cf895e-62fe-4731-84e3-d1abca163a89'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:03.646380 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b6cf895e-62fe-4731-84e3-d1abca163a89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c68b180811c4fcfb8873ca0aff606fb', 'request_id': 'req-b6cf895e-62fe-4731-84e3-d1abca163a89'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:03.657251 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b6cf895e-62fe-4731-84e3-d1abca163a89 admin admin] The token cache is being invalidate because project 9c68b180811c4fcfb8873ca0aff606fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:12:03.657480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b6cf895e-62fe-4731-84e3-d1abca163a89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:03.658176 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b6cf895e-62fe-4731-84e3-d1abca163a89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:03.659726 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 952/1905] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:12:03 2022] DELETE /identity/v3/projects/9c68b180811c4fcfb8873ca0aff606fb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:03.664666 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8fe3f-af74-4eee-99eb-eae2f69e9acc None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:03.665020 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8fe3f-af74-4eee-99eb-eae2f69e9acc None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:03.665385 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8fe3f-af74-4eee-99eb-eae2f69e9acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:03.686314 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-fbc8fe3f-af74-4eee-99eb-eae2f69e9acc None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:03.687511 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-fbc8fe3f-af74-4eee-99eb-eae2f69e9acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:03.757881 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 954/1906] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:03 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:03.765601 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607b1d3-148c-4485-9f27-9cc2811e3362 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:03.765890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607b1d3-148c-4485-9f27-9cc2811e3362 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:03.766176 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607b1d3-148c-4485-9f27-9cc2811e3362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:03.789192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3607b1d3-148c-4485-9f27-9cc2811e3362 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:03.790463 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3607b1d3-148c-4485-9f27-9cc2811e3362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:03.799319 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 953/1907] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:03 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:03.803966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc8fe3f-af74-4eee-99eb-eae2f69e9acc None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:03.814745 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc8fe3f-af74-4eee-99eb-eae2f69e9acc None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:03.819579 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045ffc5-5173-46e2-b336-6e58f504bd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:03.820533 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045ffc5-5173-46e2-b336-6e58f504bd38 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:03.820744 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045ffc5-5173-46e2-b336-6e58f504bd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:03.821029 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045ffc5-5173-46e2-b336-6e58f504bd38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:03.821782 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045ffc5-5173-46e2-b336-6e58f504bd38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:12:03.822094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045ffc5-5173-46e2-b336-6e58f504bd38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:03.824313 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045ffc5-5173-46e2-b336-6e58f504bd38 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:03.841202 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 955/1908] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:12:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:12:03.842304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607b1d3-148c-4485-9f27-9cc2811e3362 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:03.855950 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607b1d3-148c-4485-9f27-9cc2811e3362 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:03.860649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23549bb-e6ce-40e2-8e89-5769bb2df5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:03.862173 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23549bb-e6ce-40e2-8e89-5769bb2df5d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:03.862529 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23549bb-e6ce-40e2-8e89-5769bb2df5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:03.862876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23549bb-e6ce-40e2-8e89-5769bb2df5d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:03.863668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23549bb-e6ce-40e2-8e89-5769bb2df5d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:03.865654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23549bb-e6ce-40e2-8e89-5769bb2df5d1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:03.894221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 954/1909] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:12:03 2022] POST /identity/v3/projects => generated 390 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:12:03.898315 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045ffc5-5173-46e2-b336-6e58f504bd38 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:03.911999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045ffc5-5173-46e2-b336-6e58f504bd38 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:03.971091 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c6cc6-a59e-4641-b6fc-8d6eab7756c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:03.972064 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c6cc6-a59e-4641-b6fc-8d6eab7756c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:03.972064 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c6cc6-a59e-4641-b6fc-8d6eab7756c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:03.972351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c6cc6-a59e-4641-b6fc-8d6eab7756c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:03.972918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c6cc6-a59e-4641-b6fc-8d6eab7756c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:03.974920 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c6cc6-a59e-4641-b6fc-8d6eab7756c6 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:04.002120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 956/1910] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:12:03 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:12:04.006671 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23549bb-e6ce-40e2-8e89-5769bb2df5d1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:04.017274 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23549bb-e6ce-40e2-8e89-5769bb2df5d1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:04.022068 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fba420-2dfb-4dcb-b7ed-1aa3db5fa58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:04.022698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fba420-2dfb-4dcb-b7ed-1aa3db5fa58e admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:04.022985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fba420-2dfb-4dcb-b7ed-1aa3db5fa58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:04.023270 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fba420-2dfb-4dcb-b7ed-1aa3db5fa58e admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:04.024023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fba420-2dfb-4dcb-b7ed-1aa3db5fa58e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:04.025668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fba420-2dfb-4dcb-b7ed-1aa3db5fa58e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:04.040238 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 955/1911] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:04 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:04.041782 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c6cc6-a59e-4641-b6fc-8d6eab7756c6 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:04.052575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c6cc6-a59e-4641-b6fc-8d6eab7756c6 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:04.067934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18953510-a6f6-44c6-be7a-7d2c456237a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:04.068581 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18953510-a6f6-44c6-be7a-7d2c456237a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:04.068870 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18953510-a6f6-44c6-be7a-7d2c456237a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:04.069173 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18953510-a6f6-44c6-be7a-7d2c456237a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:04.069884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18953510-a6f6-44c6-be7a-7d2c456237a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:04.071495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18953510-a6f6-44c6-be7a-7d2c456237a0 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:04.086555 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 957/1912] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:04 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:04.091691 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fba420-2dfb-4dcb-b7ed-1aa3db5fa58e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:04.104285 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fba420-2dfb-4dcb-b7ed-1aa3db5fa58e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:04.109306 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c7671-33d8-4ce2-8a5f-fc1024156680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:04.110009 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c7671-33d8-4ce2-8a5f-fc1024156680 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:04.110293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c7671-33d8-4ce2-8a5f-fc1024156680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:04.110572 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c7671-33d8-4ce2-8a5f-fc1024156680 admin admin] PATH_INFO: `/v3/projects/fc249d2d6293458e881426df960f0fe2/users/9ea67da4f988486bb8ca52125180c2d4/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:04.136188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c7671-33d8-4ce2-8a5f-fc1024156680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc249d2d6293458e881426df960f0fe2, user_id=9ea67da4f988486bb8ca52125180c2d4, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:04.140313 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c7671-33d8-4ce2-8a5f-fc1024156680 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:04.167041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 956/1913] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:12:04 2022] PUT /identity/v3/projects/fc249d2d6293458e881426df960f0fe2/users/9ea67da4f988486bb8ca52125180c2d4/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:04.180353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c4bc8-4bfd-4866-a532-dc240fa59e61 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:04.182846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c4bc8-4bfd-4866-a532-dc240fa59e61 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:04.183765 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c4bc8-4bfd-4866-a532-dc240fa59e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:04.228296 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-598c4bc8-4bfd-4866-a532-dc240fa59e61 None None] MFA Rules not processed for user `9ea67da4f988486bb8ca52125180c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:04.229674 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-598c4bc8-4bfd-4866-a532-dc240fa59e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:04.234329 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:04.336487 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 958/1914] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:04 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:04.341159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c7671-33d8-4ce2-8a5f-fc1024156680 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:04.426830 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c7671-33d8-4ce2-8a5f-fc1024156680 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:04.432770 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d1427-e4f9-40a3-ac02-3738071dc783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:04.434110 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d1427-e4f9-40a3-ac02-3738071dc783 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:04.434400 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d1427-e4f9-40a3-ac02-3738071dc783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:04.434683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d1427-e4f9-40a3-ac02-3738071dc783 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:04.435599 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d1427-e4f9-40a3-ac02-3738071dc783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:04.437310 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d1427-e4f9-40a3-ac02-3738071dc783 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:04.487597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 957/1915] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:12:04 2022] POST /identity/v3/projects => generated 390 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:12:04.493272 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c4bc8-4bfd-4866-a532-dc240fa59e61 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:04.507684 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c4bc8-4bfd-4866-a532-dc240fa59e61 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:04.581040 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a94baf-17f4-4128-a7f8-a96a3a12da68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:04.581707 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a94baf-17f4-4128-a7f8-a96a3a12da68 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:04.581991 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a94baf-17f4-4128-a7f8-a96a3a12da68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:04.582271 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a94baf-17f4-4128-a7f8-a96a3a12da68 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:04.582972 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a94baf-17f4-4128-a7f8-a96a3a12da68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:04.584916 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a94baf-17f4-4128-a7f8-a96a3a12da68 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:04.609411 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 959/1916] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:12:04 2022] POST /identity/v3/users => generated 302 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:12:04.613494 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d1427-e4f9-40a3-ac02-3738071dc783 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:04.623767 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d1427-e4f9-40a3-ac02-3738071dc783 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:04.629182 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ccfe9-6010-4998-ad59-e0a4c5dc1f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:04.629806 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ccfe9-6010-4998-ad59-e0a4c5dc1f9c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:04.630095 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ccfe9-6010-4998-ad59-e0a4c5dc1f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:04.630643 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ccfe9-6010-4998-ad59-e0a4c5dc1f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:04.631378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ccfe9-6010-4998-ad59-e0a4c5dc1f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:04.632992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ccfe9-6010-4998-ad59-e0a4c5dc1f9c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:04.648085 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 958/1917] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:04 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:12:04.649638 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a94baf-17f4-4128-a7f8-a96a3a12da68 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:04.662646 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a94baf-17f4-4128-a7f8-a96a3a12da68 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:04.667660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba6b1b-6d2f-464e-8976-cf34db2af9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:04.668375 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba6b1b-6d2f-464e-8976-cf34db2af9c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:04.668660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba6b1b-6d2f-464e-8976-cf34db2af9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:04.668937 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba6b1b-6d2f-464e-8976-cf34db2af9c0 admin admin] PATH_INFO: `/v3/projects/c3495c4857684f42a0b0ed9eea766bd7/users/f79b71b5548749548c59fbfaab25a6e1/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:04.687613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba6b1b-6d2f-464e-8976-cf34db2af9c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3495c4857684f42a0b0ed9eea766bd7, user_id=f79b71b5548749548c59fbfaab25a6e1, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:04.690895 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba6b1b-6d2f-464e-8976-cf34db2af9c0 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:04.708555 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 960/1918] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:12:04 2022] PUT /identity/v3/projects/c3495c4857684f42a0b0ed9eea766bd7/users/f79b71b5548749548c59fbfaab25a6e1/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:12:04.712569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ccfe9-6010-4998-ad59-e0a4c5dc1f9c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:04.724242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ccfe9-6010-4998-ad59-e0a4c5dc1f9c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:04.765709 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:04.796200 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8c163-5002-43a4-ba4d-b42bacf9b1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:04.796946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8c163-5002-43a4-ba4d-b42bacf9b1db admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:04.797445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8c163-5002-43a4-ba4d-b42bacf9b1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:04.797794 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8c163-5002-43a4-ba4d-b42bacf9b1db admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:04.798625 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8c163-5002-43a4-ba4d-b42bacf9b1db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:04.800424 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8c163-5002-43a4-ba4d-b42bacf9b1db admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:04.812361 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 959/1919] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:04 2022] GET /identity/v3/roles => generated 1427 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:04.817442 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba6b1b-6d2f-464e-8976-cf34db2af9c0 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:04.828916 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba6b1b-6d2f-464e-8976-cf34db2af9c0 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:04.837733 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d31dfb-888d-4473-b087-344def037a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:04.838436 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d31dfb-888d-4473-b087-344def037a5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:04.838714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d31dfb-888d-4473-b087-344def037a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:04.838989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d31dfb-888d-4473-b087-344def037a5f admin admin] PATH_INFO: `/v3/projects/c3495c4857684f42a0b0ed9eea766bd7/users/f79b71b5548749548c59fbfaab25a6e1/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:04.841012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d31dfb-888d-4473-b087-344def037a5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3495c4857684f42a0b0ed9eea766bd7, user_id=f79b71b5548749548c59fbfaab25a6e1, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:04.843998 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d31dfb-888d-4473-b087-344def037a5f admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:04.859047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 961/1920] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:12:04 2022] PUT /identity/v3/projects/c3495c4857684f42a0b0ed9eea766bd7/users/f79b71b5548749548c59fbfaab25a6e1/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:04.870112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6477b-311e-4d22-9c9f-5b17279e5382 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:04.870407 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6477b-311e-4d22-9c9f-5b17279e5382 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:04.870691 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6477b-311e-4d22-9c9f-5b17279e5382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:04.904525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-26c6477b-311e-4d22-9c9f-5b17279e5382 None None] MFA Rules not processed for user `f79b71b5548749548c59fbfaab25a6e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:04.905783 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-26c6477b-311e-4d22-9c9f-5b17279e5382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:05.002764 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 960/1921] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:04 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:05.014671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d31dfb-888d-4473-b087-344def037a5f admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:05.016597 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-38d31dfb-888d-4473-b087-344def037a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:05.085848 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d31dfb-888d-4473-b087-344def037a5f admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:05.090538 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefda7bc-f748-4ffd-a693-0638628dd599 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:05.091171 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefda7bc-f748-4ffd-a693-0638628dd599 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:05.100780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefda7bc-f748-4ffd-a693-0638628dd599 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:05.100780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefda7bc-f748-4ffd-a693-0638628dd599 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:05.109138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefda7bc-f748-4ffd-a693-0638628dd599 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:05.110669 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefda7bc-f748-4ffd-a693-0638628dd599 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:05.116437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:05.128337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 962/1922] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:05 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:12:05.284842 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6477b-311e-4d22-9c9f-5b17279e5382 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:05.445498 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6477b-311e-4d22-9c9f-5b17279e5382 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:05.461014 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8ff6c-a0b6-41ab-a67b-6333ba137ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:05.461725 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8ff6c-a0b6-41ab-a67b-6333ba137ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:05.462016 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8ff6c-a0b6-41ab-a67b-6333ba137ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:05.469261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8ff6c-a0b6-41ab-a67b-6333ba137ade admin admin] PATH_INFO: `/v3/users/9ea67da4f988486bb8ca52125180c2d4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:05.469261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8ff6c-a0b6-41ab-a67b-6333ba137ade admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ea67da4f988486bb8ca52125180c2d4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:05.469261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8ff6c-a0b6-41ab-a67b-6333ba137ade admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:05.555255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ecb8ff6c-a0b6-41ab-a67b-6333ba137ade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ea67da4f988486bb8ca52125180c2d4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:05.576218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ecb8ff6c-a0b6-41ab-a67b-6333ba137ade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ea67da4f988486bb8ca52125180c2d4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:05.576680 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ecb8ff6c-a0b6-41ab-a67b-6333ba137ade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ea67da4f988486bb8ca52125180c2d4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:05.583309 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:05.592159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ecb8ff6c-a0b6-41ab-a67b-6333ba137ade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ea67da4f988486bb8ca52125180c2d4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:05.593041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ecb8ff6c-a0b6-41ab-a67b-6333ba137ade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ea67da4f988486bb8ca52125180c2d4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:05.621800 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 961/1923] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:12:05 2022] DELETE /identity/v3/users/9ea67da4f988486bb8ca52125180c2d4 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:05.624029 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4a96a-cda4-4483-a51e-da4b64ed3c75 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:05.624313 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4a96a-cda4-4483-a51e-da4b64ed3c75 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:05.624601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4a96a-cda4-4483-a51e-da4b64ed3c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:05.655788 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-96e4a96a-cda4-4483-a51e-da4b64ed3c75 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:05.657016 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-96e4a96a-cda4-4483-a51e-da4b64ed3c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:05.731765 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 963/1924] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:05.742383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8ff6c-a0b6-41ab-a67b-6333ba137ade admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:05.744471 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ecb8ff6c-a0b6-41ab-a67b-6333ba137ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:05.831913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8ff6c-a0b6-41ab-a67b-6333ba137ade admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:05.838449 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ad0c0-1e95-4a94-be14-c9552f24b3ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:05.839841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ad0c0-1e95-4a94-be14-c9552f24b3ef service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:05.840443 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ad0c0-1e95-4a94-be14-c9552f24b3ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:05.840988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ad0c0-1e95-4a94-be14-c9552f24b3ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:05.854837 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ad0c0-1e95-4a94-be14-c9552f24b3ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:05.858171 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ad0c0-1e95-4a94-be14-c9552f24b3ef service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:05.873635 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 962/1925] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:05 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:12:06.268901 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4a96a-cda4-4483-a51e-da4b64ed3c75 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:06.271328 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-96e4a96a-cda4-4483-a51e-da4b64ed3c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:06.292995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4a96a-cda4-4483-a51e-da4b64ed3c75 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:06.300218 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36c252-271a-4f86-8b87-9a6e26cac1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:06.301867 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36c252-271a-4f86-8b87-9a6e26cac1be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:06.302324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36c252-271a-4f86-8b87-9a6e26cac1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:06.302981 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36c252-271a-4f86-8b87-9a6e26cac1be admin admin] PATH_INFO: `/v3/projects/fc249d2d6293458e881426df960f0fe2` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:06.305052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36c252-271a-4f86-8b87-9a6e26cac1be admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc249d2d6293458e881426df960f0fe2)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:06.308566 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36c252-271a-4f86-8b87-9a6e26cac1be admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:06.361679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-2a36c252-271a-4f86-8b87-9a6e26cac1be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc249d2d6293458e881426df960f0fe2', 'request_id': 'req-2a36c252-271a-4f86-8b87-9a6e26cac1be'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:06.368006 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-2a36c252-271a-4f86-8b87-9a6e26cac1be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc249d2d6293458e881426df960f0fe2', 'request_id': 'req-2a36c252-271a-4f86-8b87-9a6e26cac1be'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:06.376363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-2a36c252-271a-4f86-8b87-9a6e26cac1be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc249d2d6293458e881426df960f0fe2', 'request_id': 'req-2a36c252-271a-4f86-8b87-9a6e26cac1be'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:06.386938 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-2a36c252-271a-4f86-8b87-9a6e26cac1be admin admin] The token cache is being invalidate because project fc249d2d6293458e881426df960f0fe2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:12:06.387233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-2a36c252-271a-4f86-8b87-9a6e26cac1be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:06.387874 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-2a36c252-271a-4f86-8b87-9a6e26cac1be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:06.389430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 964/1926] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:12:06 2022] DELETE /identity/v3/projects/fc249d2d6293458e881426df960f0fe2 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:06.393540 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ad0c0-1e95-4a94-be14-c9552f24b3ef service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:06.403228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ad0c0-1e95-4a94-be14-c9552f24b3ef service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:06.468288 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393d135-50aa-48ff-919e-d0e67e1e09e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:06.468864 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393d135-50aa-48ff-919e-d0e67e1e09e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:06.469180 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393d135-50aa-48ff-919e-d0e67e1e09e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:06.469462 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393d135-50aa-48ff-919e-d0e67e1e09e8 admin admin] PATH_INFO: `/v3/users/f79b71b5548749548c59fbfaab25a6e1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:06.471148 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393d135-50aa-48ff-919e-d0e67e1e09e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f79b71b5548749548c59fbfaab25a6e1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:06.473267 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393d135-50aa-48ff-919e-d0e67e1e09e8 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:06.523277 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3393d135-50aa-48ff-919e-d0e67e1e09e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f79b71b5548749548c59fbfaab25a6e1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:06.530348 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3393d135-50aa-48ff-919e-d0e67e1e09e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f79b71b5548749548c59fbfaab25a6e1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:06.531345 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3393d135-50aa-48ff-919e-d0e67e1e09e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f79b71b5548749548c59fbfaab25a6e1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:06.542584 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3393d135-50aa-48ff-919e-d0e67e1e09e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f79b71b5548749548c59fbfaab25a6e1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:06.543255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3393d135-50aa-48ff-919e-d0e67e1e09e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f79b71b5548749548c59fbfaab25a6e1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:06.559154 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:06.568002 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 963/1927] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:12:06 2022] DELETE /identity/v3/users/f79b71b5548749548c59fbfaab25a6e1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:07.375981 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36c252-271a-4f86-8b87-9a6e26cac1be admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:07.378557 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-2a36c252-271a-4f86-8b87-9a6e26cac1be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:07.409288 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36c252-271a-4f86-8b87-9a6e26cac1be admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:07.476323 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47403b5-3698-4b4c-bb88-77620410c8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:07.477012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47403b5-3698-4b4c-bb88-77620410c8a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:07.477333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47403b5-3698-4b4c-bb88-77620410c8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:07.477609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47403b5-3698-4b4c-bb88-77620410c8a4 admin admin] PATH_INFO: `/v3/projects/c3495c4857684f42a0b0ed9eea766bd7` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:07.478774 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47403b5-3698-4b4c-bb88-77620410c8a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3495c4857684f42a0b0ed9eea766bd7)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:07.480927 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47403b5-3698-4b4c-bb88-77620410c8a4 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:07.522834 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:07.527448 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393d135-50aa-48ff-919e-d0e67e1e09e8 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:07.530086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3393d135-50aa-48ff-919e-d0e67e1e09e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:07.544515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393d135-50aa-48ff-919e-d0e67e1e09e8 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:07.544725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-e47403b5-3698-4b4c-bb88-77620410c8a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3495c4857684f42a0b0ed9eea766bd7', 'request_id': 'req-e47403b5-3698-4b4c-bb88-77620410c8a4'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:07.551545 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-e47403b5-3698-4b4c-bb88-77620410c8a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3495c4857684f42a0b0ed9eea766bd7', 'request_id': 'req-e47403b5-3698-4b4c-bb88-77620410c8a4'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:07.576252 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-e47403b5-3698-4b4c-bb88-77620410c8a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3495c4857684f42a0b0ed9eea766bd7', 'request_id': 'req-e47403b5-3698-4b4c-bb88-77620410c8a4'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:07.590711 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-e47403b5-3698-4b4c-bb88-77620410c8a4 admin admin] The token cache is being invalidate because project c3495c4857684f42a0b0ed9eea766bd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:12:07.591006 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-e47403b5-3698-4b4c-bb88-77620410c8a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:07.591722 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-e47403b5-3698-4b4c-bb88-77620410c8a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:07.593320 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 965/1928] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:12:07 2022] DELETE /identity/v3/projects/c3495c4857684f42a0b0ed9eea766bd7 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:07.600285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8456b3b-d203-4560-a3c1-6eead5a52802 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:07.600581 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8456b3b-d203-4560-a3c1-6eead5a52802 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:07.600858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8456b3b-d203-4560-a3c1-6eead5a52802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:07.623736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-e8456b3b-d203-4560-a3c1-6eead5a52802 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:07.624715 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a488925-370a-4747-9d8f-553abb4ea213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:07.625285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e8456b3b-d203-4560-a3c1-6eead5a52802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:07.625858 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a488925-370a-4747-9d8f-553abb4ea213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:07.626222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a488925-370a-4747-9d8f-553abb4ea213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:07.626566 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a488925-370a-4747-9d8f-553abb4ea213 admin admin] PATH_INFO: `/v3/users/6c75f20ae1dd4984a1f770dbe944ff69` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:07.627988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a488925-370a-4747-9d8f-553abb4ea213 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c75f20ae1dd4984a1f770dbe944ff69)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:07.629955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a488925-370a-4747-9d8f-553abb4ea213 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:07.636164 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 966/1929] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:12:07.647294 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a873f-8cf9-417b-b631-cb7daf9cce91 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:07.647824 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a873f-8cf9-417b-b631-cb7daf9cce91 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:07.648736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a873f-8cf9-417b-b631-cb7daf9cce91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:07.675850 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-3c0a873f-8cf9-417b-b631-cb7daf9cce91 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:07.677095 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3c0a873f-8cf9-417b-b631-cb7daf9cce91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:07.685589 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 967/1930] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:07.689589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7a488925-370a-4747-9d8f-553abb4ea213 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c75f20ae1dd4984a1f770dbe944ff69'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:07.690044 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a873f-8cf9-417b-b631-cb7daf9cce91 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:07.697484 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7a488925-370a-4747-9d8f-553abb4ea213 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c75f20ae1dd4984a1f770dbe944ff69'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:07.698223 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7a488925-370a-4747-9d8f-553abb4ea213 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c75f20ae1dd4984a1f770dbe944ff69'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:07.700068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a873f-8cf9-417b-b631-cb7daf9cce91 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:07.706476 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5271f45-9666-4217-82aa-36b39b5f7fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:07.707371 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5271f45-9666-4217-82aa-36b39b5f7fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:07.708047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5271f45-9666-4217-82aa-36b39b5f7fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:07.708047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5271f45-9666-4217-82aa-36b39b5f7fa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:07.708753 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5271f45-9666-4217-82aa-36b39b5f7fa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:12:07.709099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5271f45-9666-4217-82aa-36b39b5f7fa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:07.711046 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5271f45-9666-4217-82aa-36b39b5f7fa8 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:07.713886 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7a488925-370a-4747-9d8f-553abb4ea213 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c75f20ae1dd4984a1f770dbe944ff69'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:07.717595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7a488925-370a-4747-9d8f-553abb4ea213 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c75f20ae1dd4984a1f770dbe944ff69'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:07.728070 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 968/1931] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:12:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:12:07.733091 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5271f45-9666-4217-82aa-36b39b5f7fa8 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:07.734938 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f5271f45-9666-4217-82aa-36b39b5f7fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:07.739111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 964/1932] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:12:07 2022] DELETE /identity/v3/users/6c75f20ae1dd4984a1f770dbe944ff69 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:12:07.743902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bd82f-f4eb-40ad-aafa-dea58cd97bd7 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:07.744764 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bd82f-f4eb-40ad-aafa-dea58cd97bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:07.744764 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bd82f-f4eb-40ad-aafa-dea58cd97bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:07.748341 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5271f45-9666-4217-82aa-36b39b5f7fa8 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:07.768812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-693bd82f-f4eb-40ad-aafa-dea58cd97bd7 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:07.770170 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-693bd82f-f4eb-40ad-aafa-dea58cd97bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:07.827084 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c388c-fd9d-4910-b03a-fe287637752a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:07.827723 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c388c-fd9d-4910-b03a-fe287637752a admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:07.828005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c388c-fd9d-4910-b03a-fe287637752a admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:07.828287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c388c-fd9d-4910-b03a-fe287637752a admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:07.828954 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c388c-fd9d-4910-b03a-fe287637752a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:07.831499 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c388c-fd9d-4910-b03a-fe287637752a admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:07.852805 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 965/1933] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:12:07.864076 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bd82f-f4eb-40ad-aafa-dea58cd97bd7 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:07.865624 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-693bd82f-f4eb-40ad-aafa-dea58cd97bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:07.884846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 969/1934] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:12:07 2022] POST /identity/v3/projects => generated 386 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:12:07.889758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c388c-fd9d-4910-b03a-fe287637752a admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:07.891706 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:07.937140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bd82f-f4eb-40ad-aafa-dea58cd97bd7 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:07.970929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c388c-fd9d-4910-b03a-fe287637752a admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:07.976065 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd072a32-cb0e-4bd4-9624-dab660cf41a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:07.976709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd072a32-cb0e-4bd4-9624-dab660cf41a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:07.976991 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd072a32-cb0e-4bd4-9624-dab660cf41a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:07.977307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd072a32-cb0e-4bd4-9624-dab660cf41a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:07.978014 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd072a32-cb0e-4bd4-9624-dab660cf41a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:07.979970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd072a32-cb0e-4bd4-9624-dab660cf41a9 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:07.993345 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de7fd1-f24e-43d1-a052-52883d7cae5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:07.993922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de7fd1-f24e-43d1-a052-52883d7cae5b service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:07.994210 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de7fd1-f24e-43d1-a052-52883d7cae5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:07.994492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de7fd1-f24e-43d1-a052-52883d7cae5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:08.006004 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de7fd1-f24e-43d1-a052-52883d7cae5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:08.007492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de7fd1-f24e-43d1-a052-52883d7cae5b service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:08.012435 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 970/1935] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:12:07 2022] POST /identity/v3/users => generated 294 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:12:08.013701 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd072a32-cb0e-4bd4-9624-dab660cf41a9 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:08.019913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 966/1936] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:07 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:12:08.020087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:08.025811 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd072a32-cb0e-4bd4-9624-dab660cf41a9 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:08.033939 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8f23f-53ae-4080-8a9a-57f94f403745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:08.035221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8f23f-53ae-4080-8a9a-57f94f403745 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:08.035899 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8f23f-53ae-4080-8a9a-57f94f403745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:08.036605 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8f23f-53ae-4080-8a9a-57f94f403745 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:08.037738 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8f23f-53ae-4080-8a9a-57f94f403745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:08.039447 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8f23f-53ae-4080-8a9a-57f94f403745 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:08.053602 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 971/1937] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:08 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:12:08.057597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de7fd1-f24e-43d1-a052-52883d7cae5b service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:08.070005 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de7fd1-f24e-43d1-a052-52883d7cae5b service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:08.075105 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa0478-a6f5-48e0-950e-9c61ca11d573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:08.075768 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa0478-a6f5-48e0-950e-9c61ca11d573 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:08.076054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa0478-a6f5-48e0-950e-9c61ca11d573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:08.076333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa0478-a6f5-48e0-950e-9c61ca11d573 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:08.077044 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa0478-a6f5-48e0-950e-9c61ca11d573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:08.079048 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa0478-a6f5-48e0-950e-9c61ca11d573 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:08.091672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 967/1938] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:08 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:08.096821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8f23f-53ae-4080-8a9a-57f94f403745 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:08.108797 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8f23f-53ae-4080-8a9a-57f94f403745 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:08.114472 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced7bb7-9651-40f1-b28e-3e0b6db889f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:08.115171 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced7bb7-9651-40f1-b28e-3e0b6db889f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:08.115601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced7bb7-9651-40f1-b28e-3e0b6db889f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:08.115887 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced7bb7-9651-40f1-b28e-3e0b6db889f6 admin admin] PATH_INFO: `/v3/projects/5ae87f308ad945eaa5515f2115d1269a/users/45453f4c81a349a0ac3682a1b2248901/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:08.140010 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced7bb7-9651-40f1-b28e-3e0b6db889f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ae87f308ad945eaa5515f2115d1269a, user_id=45453f4c81a349a0ac3682a1b2248901, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:08.143293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced7bb7-9651-40f1-b28e-3e0b6db889f6 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:08.161372 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 972/1939] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:12:08 2022] PUT /identity/v3/projects/5ae87f308ad945eaa5515f2115d1269a/users/45453f4c81a349a0ac3682a1b2248901/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:08.172898 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ee496-bf7d-4c41-a307-60dc90f6be0f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:08.173812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ee496-bf7d-4c41-a307-60dc90f6be0f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:08.174559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ee496-bf7d-4c41-a307-60dc90f6be0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:08.227066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-967ee496-bf7d-4c41-a307-60dc90f6be0f None None] MFA Rules not processed for user `45453f4c81a349a0ac3682a1b2248901`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:08.229165 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-967ee496-bf7d-4c41-a307-60dc90f6be0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:08.381333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 968/1940] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:08 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:08.399269 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced7bb7-9651-40f1-b28e-3e0b6db889f6 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:08.402233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-6ced7bb7-9651-40f1-b28e-3e0b6db889f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:08.504227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced7bb7-9651-40f1-b28e-3e0b6db889f6 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:08.512962 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d816f8a-f76b-466e-b919-13b6c0b64c13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:08.513672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d816f8a-f76b-466e-b919-13b6c0b64c13 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:08.513964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d816f8a-f76b-466e-b919-13b6c0b64c13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:08.514250 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d816f8a-f76b-466e-b919-13b6c0b64c13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:08.528104 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d816f8a-f76b-466e-b919-13b6c0b64c13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:08.529702 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d816f8a-f76b-466e-b919-13b6c0b64c13 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:08.545281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 973/1941] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:08 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:12:08.568919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ee496-bf7d-4c41-a307-60dc90f6be0f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:08.667200 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:08.677049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ee496-bf7d-4c41-a307-60dc90f6be0f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:08.682543 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843beca-9179-4fe2-884f-9811b65d81d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:08.683589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843beca-9179-4fe2-884f-9811b65d81d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:08.684049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843beca-9179-4fe2-884f-9811b65d81d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:08.687851 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843beca-9179-4fe2-884f-9811b65d81d9 admin admin] PATH_INFO: `/v3/users/45453f4c81a349a0ac3682a1b2248901` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:08.689715 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843beca-9179-4fe2-884f-9811b65d81d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45453f4c81a349a0ac3682a1b2248901)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:08.691864 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843beca-9179-4fe2-884f-9811b65d81d9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:08.780521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4843beca-9179-4fe2-884f-9811b65d81d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45453f4c81a349a0ac3682a1b2248901'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:08.789922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4843beca-9179-4fe2-884f-9811b65d81d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45453f4c81a349a0ac3682a1b2248901'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:08.790826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4843beca-9179-4fe2-884f-9811b65d81d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45453f4c81a349a0ac3682a1b2248901'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:08.810982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4843beca-9179-4fe2-884f-9811b65d81d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45453f4c81a349a0ac3682a1b2248901'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:08.811831 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4843beca-9179-4fe2-884f-9811b65d81d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45453f4c81a349a0ac3682a1b2248901'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:08.843050 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 969/1942] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:12:08 2022] DELETE /identity/v3/users/45453f4c81a349a0ac3682a1b2248901 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:08.844480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb31b0b-7d15-4ac7-8b41-5a390576cb8d None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:08.844990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb31b0b-7d15-4ac7-8b41-5a390576cb8d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:08.845370 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb31b0b-7d15-4ac7-8b41-5a390576cb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:08.870528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-edb31b0b-7d15-4ac7-8b41-5a390576cb8d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:08.872314 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-edb31b0b-7d15-4ac7-8b41-5a390576cb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:08.906856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843beca-9179-4fe2-884f-9811b65d81d9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:08.909703 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4843beca-9179-4fe2-884f-9811b65d81d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:09.068265 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 974/1943] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:09.073206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843beca-9179-4fe2-884f-9811b65d81d9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:09.080785 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3aa37-854f-4394-a5d0-2a8cd18e8336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:09.080785 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3aa37-854f-4394-a5d0-2a8cd18e8336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:09.081075 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3aa37-854f-4394-a5d0-2a8cd18e8336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:09.081852 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3aa37-854f-4394-a5d0-2a8cd18e8336 admin admin] PATH_INFO: `/v3/projects/3c1f7edebdbc471fbef44e7833ef1cef` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:09.083133 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3aa37-854f-4394-a5d0-2a8cd18e8336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c1f7edebdbc471fbef44e7833ef1cef)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:09.084876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb31b0b-7d15-4ac7-8b41-5a390576cb8d None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:09.085315 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3aa37-854f-4394-a5d0-2a8cd18e8336 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:09.090329 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-edb31b0b-7d15-4ac7-8b41-5a390576cb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:09.150771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9fa3aa37-854f-4394-a5d0-2a8cd18e8336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c1f7edebdbc471fbef44e7833ef1cef', 'request_id': 'req-9fa3aa37-854f-4394-a5d0-2a8cd18e8336'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:09.159804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9fa3aa37-854f-4394-a5d0-2a8cd18e8336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c1f7edebdbc471fbef44e7833ef1cef', 'request_id': 'req-9fa3aa37-854f-4394-a5d0-2a8cd18e8336'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:09.180549 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9fa3aa37-854f-4394-a5d0-2a8cd18e8336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c1f7edebdbc471fbef44e7833ef1cef', 'request_id': 'req-9fa3aa37-854f-4394-a5d0-2a8cd18e8336'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:09.197628 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9fa3aa37-854f-4394-a5d0-2a8cd18e8336 admin admin] The token cache is being invalidate because project 3c1f7edebdbc471fbef44e7833ef1cef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:12:09.197962 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9fa3aa37-854f-4394-a5d0-2a8cd18e8336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:09.198590 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9fa3aa37-854f-4394-a5d0-2a8cd18e8336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:09.199482 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb31b0b-7d15-4ac7-8b41-5a390576cb8d None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:09.200190 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 970/1944] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:12:08 2022] DELETE /identity/v3/projects/3c1f7edebdbc471fbef44e7833ef1cef => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:09.201330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3aa37-854f-4394-a5d0-2a8cd18e8336 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:09.205312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9fa3aa37-854f-4394-a5d0-2a8cd18e8336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:09.218577 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:09.239704 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:09.274475 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4af722-5af1-4067-b9de-2d3cb1a9df40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:09.275126 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4af722-5af1-4067-b9de-2d3cb1a9df40 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:09.275409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4af722-5af1-4067-b9de-2d3cb1a9df40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:09.275683 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4af722-5af1-4067-b9de-2d3cb1a9df40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:09.279100 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ba4af722-5af1-4067-b9de-2d3cb1a9df40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:09.286725 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3aa37-854f-4394-a5d0-2a8cd18e8336 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:09.289804 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4af722-5af1-4067-b9de-2d3cb1a9df40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:09.293087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4af722-5af1-4067-b9de-2d3cb1a9df40 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:09.293254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6671675a-41e9-4d89-87dc-6816ccafcd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:09.293254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671675a-41e9-4d89-87dc-6816ccafcd05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:09.293254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671675a-41e9-4d89-87dc-6816ccafcd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:09.293254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671675a-41e9-4d89-87dc-6816ccafcd05 admin admin] PATH_INFO: `/v3/users/84b7ddd2dcf14086bd855fa288b9451c` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:09.294397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6671675a-41e9-4d89-87dc-6816ccafcd05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84b7ddd2dcf14086bd855fa288b9451c)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:09.296450 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6671675a-41e9-4d89-87dc-6816ccafcd05 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:09.301947 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 975/1945] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:09 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:12:09.303165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6977fd-ac21-4cc0-83c8-094725fb7818 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:09.303454 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6977fd-ac21-4cc0-83c8-094725fb7818 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:09.303734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6977fd-ac21-4cc0-83c8-094725fb7818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:09.328797 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-8d6977fd-ac21-4cc0-83c8-094725fb7818 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:09.330684 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-8d6977fd-ac21-4cc0-83c8-094725fb7818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:09.340286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 976/1946] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:09.349680 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd909f-936f-4841-b8bd-832b9c629516 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:09.350069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd909f-936f-4841-b8bd-832b9c629516 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:09.350655 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd909f-936f-4841-b8bd-832b9c629516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:09.371567 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6671675a-41e9-4d89-87dc-6816ccafcd05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84b7ddd2dcf14086bd855fa288b9451c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:09.374144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-3fcd909f-936f-4841-b8bd-832b9c629516 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:09.375641 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3fcd909f-936f-4841-b8bd-832b9c629516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:09.387010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6671675a-41e9-4d89-87dc-6816ccafcd05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84b7ddd2dcf14086bd855fa288b9451c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:09.388331 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6671675a-41e9-4d89-87dc-6816ccafcd05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84b7ddd2dcf14086bd855fa288b9451c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:09.390430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 977/1947] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:12:09.395391 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd909f-936f-4841-b8bd-832b9c629516 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:09.405419 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd909f-936f-4841-b8bd-832b9c629516 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:09.414921 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e3204-01c3-4f50-b676-3c98ccea4e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:09.415729 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6671675a-41e9-4d89-87dc-6816ccafcd05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84b7ddd2dcf14086bd855fa288b9451c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:09.416061 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e3204-01c3-4f50-b676-3c98ccea4e03 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:09.416355 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e3204-01c3-4f50-b676-3c98ccea4e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:09.416700 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e3204-01c3-4f50-b676-3c98ccea4e03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:09.417663 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e3204-01c3-4f50-b676-3c98ccea4e03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:12:09.417983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e3204-01c3-4f50-b676-3c98ccea4e03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:09.419920 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e3204-01c3-4f50-b676-3c98ccea4e03 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:09.426358 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6671675a-41e9-4d89-87dc-6816ccafcd05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84b7ddd2dcf14086bd855fa288b9451c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:09.435306 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 978/1948] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:12:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:12:09.440823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e3204-01c3-4f50-b676-3c98ccea4e03 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:09.442648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-7e5e3204-01c3-4f50-b676-3c98ccea4e03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:09.472264 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e3204-01c3-4f50-b676-3c98ccea4e03 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:09.489782 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030495e-df84-47c6-8c32-b5f9d28a1601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:09.490424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030495e-df84-47c6-8c32-b5f9d28a1601 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:09.490710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030495e-df84-47c6-8c32-b5f9d28a1601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:09.490984 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030495e-df84-47c6-8c32-b5f9d28a1601 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:09.491853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d030495e-df84-47c6-8c32-b5f9d28a1601 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:09.493907 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d030495e-df84-47c6-8c32-b5f9d28a1601 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:09.518880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 971/1949] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:12:09 2022] DELETE /identity/v3/users/84b7ddd2dcf14086bd855fa288b9451c => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:09.524581 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95ff93-841c-457a-b9f1-58cbf22487e9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:09.525956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95ff93-841c-457a-b9f1-58cbf22487e9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:09.526330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95ff93-841c-457a-b9f1-58cbf22487e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:09.547279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 979/1950] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:12:09 2022] POST /identity/v3/projects => generated 366 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:12:09.552726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030495e-df84-47c6-8c32-b5f9d28a1601 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:09.580028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5f95ff93-841c-457a-b9f1-58cbf22487e9 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:09.581376 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5f95ff93-841c-457a-b9f1-58cbf22487e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:09.643997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030495e-df84-47c6-8c32-b5f9d28a1601 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:09.650868 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bd414-bfcf-4439-a720-d08e6227aad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:09.651520 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bd414-bfcf-4439-a720-d08e6227aad0 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:09.651807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bd414-bfcf-4439-a720-d08e6227aad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:09.652091 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bd414-bfcf-4439-a720-d08e6227aad0 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:09.652967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390bd414-bfcf-4439-a720-d08e6227aad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:09.655319 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390bd414-bfcf-4439-a720-d08e6227aad0 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:09.692459 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 980/1951] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:12:09 2022] POST /identity/v3/users => generated 284 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:12:09.694328 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bd414-bfcf-4439-a720-d08e6227aad0 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:09.707242 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bd414-bfcf-4439-a720-d08e6227aad0 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:09.712227 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 972/1952] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:09.716659 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58725408-181e-4bce-bc8b-83ca271c4d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:09.717443 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58725408-181e-4bce-bc8b-83ca271c4d39 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:09.717848 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58725408-181e-4bce-bc8b-83ca271c4d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:09.718228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58725408-181e-4bce-bc8b-83ca271c4d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:09.719107 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58725408-181e-4bce-bc8b-83ca271c4d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:09.721806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58725408-181e-4bce-bc8b-83ca271c4d39 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:09.734454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95ff93-841c-457a-b9f1-58cbf22487e9 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:09.736136 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5f95ff93-841c-457a-b9f1-58cbf22487e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:09.739849 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 981/1953] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:09 2022] GET /identity/v3/roles => generated 1427 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:09.740972 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58725408-181e-4bce-bc8b-83ca271c4d39 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:09.758109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58725408-181e-4bce-bc8b-83ca271c4d39 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:09.766128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a98b6-b540-4103-8b33-c6134069a7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:09.766128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a98b6-b540-4103-8b33-c6134069a7ce admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:09.766128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a98b6-b540-4103-8b33-c6134069a7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:09.766128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a98b6-b540-4103-8b33-c6134069a7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:09.766647 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a98b6-b540-4103-8b33-c6134069a7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:09.768037 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a98b6-b540-4103-8b33-c6134069a7ce admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:09.783768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 982/1954] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:09 2022] GET /identity/v3/roles => generated 1427 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:09.785764 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:09.788110 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a98b6-b540-4103-8b33-c6134069a7ce admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:09.804846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a98b6-b540-4103-8b33-c6134069a7ce admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:09.811019 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c28b2-483d-4237-9291-960537e5d776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:09.812950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c28b2-483d-4237-9291-960537e5d776 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:09.814962 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c28b2-483d-4237-9291-960537e5d776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:09.823762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c28b2-483d-4237-9291-960537e5d776 admin admin] PATH_INFO: `/v3/projects/e250298a849145b9b7844caf5205c85d/users/14f3f9dd0bd0428a8143c6ef6568635c/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:09.851577 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95ff93-841c-457a-b9f1-58cbf22487e9 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:09.859180 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c28b2-483d-4237-9291-960537e5d776 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e250298a849145b9b7844caf5205c85d, user_id=14f3f9dd0bd0428a8143c6ef6568635c, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:09.859639 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c05b92-27a8-4202-8285-8e244c15fc68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:09.860300 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c05b92-27a8-4202-8285-8e244c15fc68 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:09.860636 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c05b92-27a8-4202-8285-8e244c15fc68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:09.860919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c05b92-27a8-4202-8285-8e244c15fc68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:09.862495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c28b2-483d-4237-9291-960537e5d776 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:09.876850 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c05b92-27a8-4202-8285-8e244c15fc68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:09.878858 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c05b92-27a8-4202-8285-8e244c15fc68 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:09.897301 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 983/1955] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:12:09 2022] PUT /identity/v3/projects/e250298a849145b9b7844caf5205c85d/users/14f3f9dd0bd0428a8143c6ef6568635c/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:09.905872 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 973/1956] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:09 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:12:09.909602 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f96af-57bf-401e-b07d-4067804500df None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:09.910039 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f96af-57bf-401e-b07d-4067804500df None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:09.910419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f96af-57bf-401e-b07d-4067804500df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:09.917034 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c28b2-483d-4237-9291-960537e5d776 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:09.925192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-148c28b2-483d-4237-9291-960537e5d776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:09.975563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-669f96af-57bf-401e-b07d-4067804500df None None] MFA Rules not processed for user `14f3f9dd0bd0428a8143c6ef6568635c`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:09.976571 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:09.977061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-669f96af-57bf-401e-b07d-4067804500df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:10.035698 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c28b2-483d-4237-9291-960537e5d776 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:10.062967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b345d71-4428-40f5-bf7f-97d15c643e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:10.064923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b345d71-4428-40f5-bf7f-97d15c643e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:10.065351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b345d71-4428-40f5-bf7f-97d15c643e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:10.065640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b345d71-4428-40f5-bf7f-97d15c643e12 admin admin] PATH_INFO: `/v3/projects/5ae87f308ad945eaa5515f2115d1269a` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:10.067277 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b345d71-4428-40f5-bf7f-97d15c643e12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ae87f308ad945eaa5515f2115d1269a)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:10.073256 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b345d71-4428-40f5-bf7f-97d15c643e12 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:10.105532 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 974/1957] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:10.117959 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f96af-57bf-401e-b07d-4067804500df None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:10.132116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-669f96af-57bf-401e-b07d-4067804500df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:10.187475 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-9b345d71-4428-40f5-bf7f-97d15c643e12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ae87f308ad945eaa5515f2115d1269a', 'request_id': 'req-9b345d71-4428-40f5-bf7f-97d15c643e12'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:10.197374 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-9b345d71-4428-40f5-bf7f-97d15c643e12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ae87f308ad945eaa5515f2115d1269a', 'request_id': 'req-9b345d71-4428-40f5-bf7f-97d15c643e12'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:10.213367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-9b345d71-4428-40f5-bf7f-97d15c643e12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ae87f308ad945eaa5515f2115d1269a', 'request_id': 'req-9b345d71-4428-40f5-bf7f-97d15c643e12'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:10.232137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-9b345d71-4428-40f5-bf7f-97d15c643e12 admin admin] The token cache is being invalidate because project 5ae87f308ad945eaa5515f2115d1269a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:12:10.233209 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-9b345d71-4428-40f5-bf7f-97d15c643e12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:10.235392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-9b345d71-4428-40f5-bf7f-97d15c643e12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:10.243205 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 984/1958] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:12:09 2022] DELETE /identity/v3/projects/5ae87f308ad945eaa5515f2115d1269a => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:10.243425 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f96af-57bf-401e-b07d-4067804500df None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:10.244959 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba19d0-a9a5-4301-bac6-53c5d9aa2e64 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:10.245297 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba19d0-a9a5-4301-bac6-53c5d9aa2e64 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:10.245581 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba19d0-a9a5-4301-bac6-53c5d9aa2e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:10.272456 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-6bba19d0-a9a5-4301-bac6-53c5d9aa2e64 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:10.275231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-6bba19d0-a9a5-4301-bac6-53c5d9aa2e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:10.324304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6be4e9-4992-4e1c-950f-4e64dcb2af3f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd959ddd087724547a11c3d4c796d8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:10.325249 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6be4e9-4992-4e1c-950f-4e64dcb2af3f service glance] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:10.325717 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6be4e9-4992-4e1c-950f-4e64dcb2af3f service glance] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:10.326155 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6be4e9-4992-4e1c-950f-4e64dcb2af3f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:10.328693 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6b6be4e9-4992-4e1c-950f-4e64dcb2af3f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:10.374461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:10.377512 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 985/1959] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:10.386341 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3cdab-9766-475b-8ba6-ec04bb637283 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:10.386633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3cdab-9766-475b-8ba6-ec04bb637283 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:10.386918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3cdab-9766-475b-8ba6-ec04bb637283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:10.411966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-d3f3cdab-9766-475b-8ba6-ec04bb637283 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:10.413431 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d3f3cdab-9766-475b-8ba6-ec04bb637283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:10.416775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6be4e9-4992-4e1c-950f-4e64dcb2af3f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:10.418647 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6be4e9-4992-4e1c-950f-4e64dcb2af3f service glance] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:10.431212 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 986/1960] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:12:10.431585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 975/1961] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:12:10.435555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6be4e9-4992-4e1c-950f-4e64dcb2af3f service glance] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:10.448511 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6be4e9-4992-4e1c-950f-4e64dcb2af3f service glance] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:10.456016 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b73e93-6cc2-410a-8c2a-03b75987bad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:10.456692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b73e93-6cc2-410a-8c2a-03b75987bad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:10.457055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b73e93-6cc2-410a-8c2a-03b75987bad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:10.457375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b73e93-6cc2-410a-8c2a-03b75987bad2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:10.458088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b73e93-6cc2-410a-8c2a-03b75987bad2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:12:10.458402 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b73e93-6cc2-410a-8c2a-03b75987bad2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:10.460029 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b73e93-6cc2-410a-8c2a-03b75987bad2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:10.473739 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 976/1962] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:12:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:12:10.477628 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3cdab-9766-475b-8ba6-ec04bb637283 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:10.488475 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3cdab-9766-475b-8ba6-ec04bb637283 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:10.502790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c19239-799d-4237-a8c2-5c0ea2ea5083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:10.503719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c19239-799d-4237-a8c2-5c0ea2ea5083 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:10.504104 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c19239-799d-4237-a8c2-5c0ea2ea5083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:10.505679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c19239-799d-4237-a8c2-5c0ea2ea5083 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:10.506461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c19239-799d-4237-a8c2-5c0ea2ea5083 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:10.509004 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c19239-799d-4237-a8c2-5c0ea2ea5083 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:10.550847 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b73e93-6cc2-410a-8c2a-03b75987bad2 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:10.554580 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-39b73e93-6cc2-410a-8c2a-03b75987bad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:10.576033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b73e93-6cc2-410a-8c2a-03b75987bad2 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:10.581280 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3ec65-9bd5-4ec6-8b28-28aadd7bedfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:10.582174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3ec65-9bd5-4ec6-8b28-28aadd7bedfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:10.582454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3ec65-9bd5-4ec6-8b28-28aadd7bedfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:10.582759 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3ec65-9bd5-4ec6-8b28-28aadd7bedfd admin admin] PATH_INFO: `/v3/projects/ec82b9b11f544b70977cf477be932c91` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:10.584713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb3ec65-9bd5-4ec6-8b28-28aadd7bedfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec82b9b11f544b70977cf477be932c91)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:10.586848 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb3ec65-9bd5-4ec6-8b28-28aadd7bedfd admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:10.605225 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 987/1963] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:12:10 2022] POST /identity/v3/projects => generated 376 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:12:10.609026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c19239-799d-4237-a8c2-5c0ea2ea5083 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:10.652022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:10.656129 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ebb3ec65-9bd5-4ec6-8b28-28aadd7bedfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec82b9b11f544b70977cf477be932c91', 'request_id': 'req-ebb3ec65-9bd5-4ec6-8b28-28aadd7bedfd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:10.674783 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ebb3ec65-9bd5-4ec6-8b28-28aadd7bedfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec82b9b11f544b70977cf477be932c91', 'request_id': 'req-ebb3ec65-9bd5-4ec6-8b28-28aadd7bedfd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:10.686406 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ebb3ec65-9bd5-4ec6-8b28-28aadd7bedfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec82b9b11f544b70977cf477be932c91', 'request_id': 'req-ebb3ec65-9bd5-4ec6-8b28-28aadd7bedfd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:10.700507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ebb3ec65-9bd5-4ec6-8b28-28aadd7bedfd admin admin] The token cache is being invalidate because project ec82b9b11f544b70977cf477be932c91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:12:10.700803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ebb3ec65-9bd5-4ec6-8b28-28aadd7bedfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:10.701546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ebb3ec65-9bd5-4ec6-8b28-28aadd7bedfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:10.701905 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c19239-799d-4237-a8c2-5c0ea2ea5083 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:10.703210 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 977/1964] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:12:10 2022] DELETE /identity/v3/projects/ec82b9b11f544b70977cf477be932c91 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:10.707741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3ec65-9bd5-4ec6-8b28-28aadd7bedfd admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:10.709902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ebb3ec65-9bd5-4ec6-8b28-28aadd7bedfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:10.776557 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e391c-7e21-4ca1-bc39-a8018c61bdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:10.777413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e391c-7e21-4ca1-bc39-a8018c61bdfd admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:10.777794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e391c-7e21-4ca1-bc39-a8018c61bdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:10.778149 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e391c-7e21-4ca1-bc39-a8018c61bdfd admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:10.778902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e391c-7e21-4ca1-bc39-a8018c61bdfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:10.781004 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e391c-7e21-4ca1-bc39-a8018c61bdfd admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:10.806041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3ec65-9bd5-4ec6-8b28-28aadd7bedfd admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:10.810834 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 988/1965] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:12:10 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:12:10.811340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d750f4-31db-4717-a20c-4d5bbf930253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:10.812006 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d750f4-31db-4717-a20c-4d5bbf930253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:10.812225 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e391c-7e21-4ca1-bc39-a8018c61bdfd admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:10.812327 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d750f4-31db-4717-a20c-4d5bbf930253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:10.812548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d750f4-31db-4717-a20c-4d5bbf930253 admin admin] PATH_INFO: `/v3/users/062391805ac4421ab679e69202b6702f` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:10.815531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d750f4-31db-4717-a20c-4d5bbf930253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=062391805ac4421ab679e69202b6702f)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:10.815784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-6b9e391c-7e21-4ca1-bc39-a8018c61bdfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:10.817476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d750f4-31db-4717-a20c-4d5bbf930253 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:10.828384 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e391c-7e21-4ca1-bc39-a8018c61bdfd admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:10.834613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77e748-0503-4f92-a049-0b2d920d3c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:10.835265 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77e748-0503-4f92-a049-0b2d920d3c27 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:10.835554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77e748-0503-4f92-a049-0b2d920d3c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:10.835835 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77e748-0503-4f92-a049-0b2d920d3c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:10.836550 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77e748-0503-4f92-a049-0b2d920d3c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:10.838282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77e748-0503-4f92-a049-0b2d920d3c27 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:10.855318 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 989/1966] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:10 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:10.859595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77e748-0503-4f92-a049-0b2d920d3c27 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:10.871481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77e748-0503-4f92-a049-0b2d920d3c27 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:10.873617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:10.877943 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abee9e0-7e0e-4f5a-a755-af7344172994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:10.878701 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abee9e0-7e0e-4f5a-a755-af7344172994 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:10.878987 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abee9e0-7e0e-4f5a-a755-af7344172994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:10.879407 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abee9e0-7e0e-4f5a-a755-af7344172994 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:10.880124 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abee9e0-7e0e-4f5a-a755-af7344172994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:10.882114 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abee9e0-7e0e-4f5a-a755-af7344172994 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:10.883540 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a9d750f4-31db-4717-a20c-4d5bbf930253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '062391805ac4421ab679e69202b6702f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:10.893985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a9d750f4-31db-4717-a20c-4d5bbf930253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '062391805ac4421ab679e69202b6702f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:10.895192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a9d750f4-31db-4717-a20c-4d5bbf930253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '062391805ac4421ab679e69202b6702f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:10.898048 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 990/1967] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:10 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:12:10.902814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abee9e0-7e0e-4f5a-a755-af7344172994 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:10.913779 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a9d750f4-31db-4717-a20c-4d5bbf930253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '062391805ac4421ab679e69202b6702f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:10.917226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abee9e0-7e0e-4f5a-a755-af7344172994 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:10.917834 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a9d750f4-31db-4717-a20c-4d5bbf930253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '062391805ac4421ab679e69202b6702f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:10.923979 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc78b6-2ef0-47c1-b982-9fdc48019dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:10.924101 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc78b6-2ef0-47c1-b982-9fdc48019dd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:10.924406 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc78b6-2ef0-47c1-b982-9fdc48019dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:10.924691 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc78b6-2ef0-47c1-b982-9fdc48019dd3 admin admin] PATH_INFO: `/v3/projects/11cb7b53383b4f708c9d508e90072d20/users/f08fe9a161dd4757a70d4c2075436901/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:10.959225 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc78b6-2ef0-47c1-b982-9fdc48019dd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11cb7b53383b4f708c9d508e90072d20, user_id=f08fe9a161dd4757a70d4c2075436901, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:10.961906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 978/1968] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:12:10 2022] DELETE /identity/v3/users/062391805ac4421ab679e69202b6702f => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:12:10.962544 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc78b6-2ef0-47c1-b982-9fdc48019dd3 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:10.983866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 991/1969] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:12:10 2022] PUT /identity/v3/projects/11cb7b53383b4f708c9d508e90072d20/users/f08fe9a161dd4757a70d4c2075436901/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:10.997143 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1facc2-398d-4d5d-a98f-86417df9e3a0 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:10.997447 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1facc2-398d-4d5d-a98f-86417df9e3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:10.997733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1facc2-398d-4d5d-a98f-86417df9e3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:11.074477 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8a1facc2-398d-4d5d-a98f-86417df9e3a0 None None] MFA Rules not processed for user `f08fe9a161dd4757a70d4c2075436901`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:11.075849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8a1facc2-398d-4d5d-a98f-86417df9e3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:11.187205 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 979/1970] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:11.198839 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc78b6-2ef0-47c1-b982-9fdc48019dd3 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:11.202925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3fdc78b6-2ef0-47c1-b982-9fdc48019dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:11.292240 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc78b6-2ef0-47c1-b982-9fdc48019dd3 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:11.297923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6344cb-7ed9-4881-9223-ccc73fcd9bb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:11.299688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6344cb-7ed9-4881-9223-ccc73fcd9bb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:11.300006 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6344cb-7ed9-4881-9223-ccc73fcd9bb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:11.300267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6344cb-7ed9-4881-9223-ccc73fcd9bb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:11.316665 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6344cb-7ed9-4881-9223-ccc73fcd9bb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:11.318438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6344cb-7ed9-4881-9223-ccc73fcd9bb9 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:11.341701 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 992/1971] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:12:11.529841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1facc2-398d-4d5d-a98f-86417df9e3a0 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:11.533181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8a1facc2-398d-4d5d-a98f-86417df9e3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:11.658825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1facc2-398d-4d5d-a98f-86417df9e3a0 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:11.677578 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ae6d4-6bb1-4a8a-98d9-f2eca8ac5527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:11.678096 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ae6d4-6bb1-4a8a-98d9-f2eca8ac5527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:11.678378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ae6d4-6bb1-4a8a-98d9-f2eca8ac5527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:11.678659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ae6d4-6bb1-4a8a-98d9-f2eca8ac5527 admin admin] PATH_INFO: `/v3/projects/394f34a75f2d406382254e013f03838f` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:11.681358 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ae6d4-6bb1-4a8a-98d9-f2eca8ac5527 admin admin] RBAC: Authorizing `identity:delete_project(project_id=394f34a75f2d406382254e013f03838f)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:11.683449 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ae6d4-6bb1-4a8a-98d9-f2eca8ac5527 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:11.752728 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d48ae6d4-6bb1-4a8a-98d9-f2eca8ac5527 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '394f34a75f2d406382254e013f03838f', 'request_id': 'req-d48ae6d4-6bb1-4a8a-98d9-f2eca8ac5527'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:11.754092 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:11.761880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d48ae6d4-6bb1-4a8a-98d9-f2eca8ac5527 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '394f34a75f2d406382254e013f03838f', 'request_id': 'req-d48ae6d4-6bb1-4a8a-98d9-f2eca8ac5527'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:11.771165 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d48ae6d4-6bb1-4a8a-98d9-f2eca8ac5527 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '394f34a75f2d406382254e013f03838f', 'request_id': 'req-d48ae6d4-6bb1-4a8a-98d9-f2eca8ac5527'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:11.782263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d48ae6d4-6bb1-4a8a-98d9-f2eca8ac5527 admin admin] The token cache is being invalidate because project 394f34a75f2d406382254e013f03838f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:12:11.783104 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d48ae6d4-6bb1-4a8a-98d9-f2eca8ac5527 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:11.783927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d48ae6d4-6bb1-4a8a-98d9-f2eca8ac5527 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:11.785516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 980/1972] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:12:11 2022] DELETE /identity/v3/projects/394f34a75f2d406382254e013f03838f => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:11.790349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9ba4a-c42c-489b-865b-3e5390e64e52 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:11.790847 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9ba4a-c42c-489b-865b-3e5390e64e52 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:11.790945 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9ba4a-c42c-489b-865b-3e5390e64e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:11.810785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-26f9ba4a-c42c-489b-865b-3e5390e64e52 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:11.811985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-26f9ba4a-c42c-489b-865b-3e5390e64e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:11.829802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:11.877446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 993/1973] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:11.886237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f90ca-d31c-494d-8837-a8e272318b68 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:11.886530 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f90ca-d31c-494d-8837-a8e272318b68 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:11.886812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f90ca-d31c-494d-8837-a8e272318b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:11.908601 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-449f90ca-d31c-494d-8837-a8e272318b68 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:11.909852 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-449f90ca-d31c-494d-8837-a8e272318b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:11.920471 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 981/1974] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:11.925474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9ba4a-c42c-489b-865b-3e5390e64e52 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:11.936707 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9ba4a-c42c-489b-865b-3e5390e64e52 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:11.941933 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1518a8-c304-4984-ade2-8e2255793a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:11.943157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1518a8-c304-4984-ade2-8e2255793a82 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:11.943157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1518a8-c304-4984-ade2-8e2255793a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:11.943157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1518a8-c304-4984-ade2-8e2255793a82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:11.943832 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1518a8-c304-4984-ade2-8e2255793a82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:12:11.944172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1518a8-c304-4984-ade2-8e2255793a82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:11.945837 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1518a8-c304-4984-ade2-8e2255793a82 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:11.959964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 994/1975] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:12:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:12:11.964751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f90ca-d31c-494d-8837-a8e272318b68 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:11.975148 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f90ca-d31c-494d-8837-a8e272318b68 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:11.979856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de4f40-5083-4798-b979-a19590b4759b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:11.980481 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de4f40-5083-4798-b979-a19590b4759b admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:11.980770 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de4f40-5083-4798-b979-a19590b4759b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:11.981048 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de4f40-5083-4798-b979-a19590b4759b admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:11.981826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de4f40-5083-4798-b979-a19590b4759b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:11.983737 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de4f40-5083-4798-b979-a19590b4759b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:12.012761 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 982/1976] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:12:11 2022] POST /identity/v3/projects => generated 376 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:12:12.018248 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1518a8-c304-4984-ade2-8e2255793a82 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:12.028643 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1518a8-c304-4984-ade2-8e2255793a82 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:12.112450 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0517df6-9974-44db-92b8-695ab640d5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:12.113397 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0517df6-9974-44db-92b8-695ab640d5af admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:12.113822 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0517df6-9974-44db-92b8-695ab640d5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:12.114266 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0517df6-9974-44db-92b8-695ab640d5af admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:12.114950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0517df6-9974-44db-92b8-695ab640d5af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:12.116882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0517df6-9974-44db-92b8-695ab640d5af admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:12.143846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 995/1977] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:12:12 2022] POST /identity/v3/users => generated 289 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:12:12.156093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de4f40-5083-4798-b979-a19590b4759b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:12.165895 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de4f40-5083-4798-b979-a19590b4759b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:12.170686 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45cf40-a076-44ef-907b-8ce98aff8f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:12.171323 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45cf40-a076-44ef-907b-8ce98aff8f5f admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:12.171611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45cf40-a076-44ef-907b-8ce98aff8f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:12.171892 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45cf40-a076-44ef-907b-8ce98aff8f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:12.172593 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45cf40-a076-44ef-907b-8ce98aff8f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:12.174447 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45cf40-a076-44ef-907b-8ce98aff8f5f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:12.192667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 983/1978] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:12 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:12.194643 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0517df6-9974-44db-92b8-695ab640d5af admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:12.206732 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0517df6-9974-44db-92b8-695ab640d5af admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:12.212109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c7da3-de45-4de9-ad2c-85257dd9e2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:12.212772 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c7da3-de45-4de9-ad2c-85257dd9e2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:12.213057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c7da3-de45-4de9-ad2c-85257dd9e2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:12.213385 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c7da3-de45-4de9-ad2c-85257dd9e2e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:12.214088 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c7da3-de45-4de9-ad2c-85257dd9e2e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:12.215804 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c7da3-de45-4de9-ad2c-85257dd9e2e0 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:12.228762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 996/1979] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:12 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:12.232851 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45cf40-a076-44ef-907b-8ce98aff8f5f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:12.244914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45cf40-a076-44ef-907b-8ce98aff8f5f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:12.254051 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994e1eb-c68e-4695-85ae-7c693afb8be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:12.254855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994e1eb-c68e-4695-85ae-7c693afb8be5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:12.255477 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994e1eb-c68e-4695-85ae-7c693afb8be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:12.255838 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994e1eb-c68e-4695-85ae-7c693afb8be5 admin admin] PATH_INFO: `/v3/projects/7e0e5bb1c55c457899677b3043941866/users/bbf2980dae53403f99f4661ed9078368/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:12.276192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994e1eb-c68e-4695-85ae-7c693afb8be5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e0e5bb1c55c457899677b3043941866, user_id=bbf2980dae53403f99f4661ed9078368, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:12.279101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994e1eb-c68e-4695-85ae-7c693afb8be5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:12.300707 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 984/1980] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:12:12 2022] PUT /identity/v3/projects/7e0e5bb1c55c457899677b3043941866/users/bbf2980dae53403f99f4661ed9078368/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:12.311271 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee333e-2770-443a-a18f-54020b3f4993 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:12.311271 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee333e-2770-443a-a18f-54020b3f4993 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:12.311575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee333e-2770-443a-a18f-54020b3f4993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:12.346744 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-b6ee333e-2770-443a-a18f-54020b3f4993 None None] MFA Rules not processed for user `bbf2980dae53403f99f4661ed9078368`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:12.347969 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b6ee333e-2770-443a-a18f-54020b3f4993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:12.427097 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 997/1981] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:12.432218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994e1eb-c68e-4695-85ae-7c693afb8be5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:12.508632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994e1eb-c68e-4695-85ae-7c693afb8be5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:12.513496 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242c682-0f93-456d-aa07-1aca5c302d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:12.514125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242c682-0f93-456d-aa07-1aca5c302d4c admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:12.514407 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242c682-0f93-456d-aa07-1aca5c302d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:12.514682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242c682-0f93-456d-aa07-1aca5c302d4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:12.515366 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242c682-0f93-456d-aa07-1aca5c302d4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:12.518239 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242c682-0f93-456d-aa07-1aca5c302d4c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:12.556593 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 985/1982] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:12:12 2022] POST /identity/v3/projects => generated 376 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:12:12.558042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee333e-2770-443a-a18f-54020b3f4993 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:12.570261 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee333e-2770-443a-a18f-54020b3f4993 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:12.586157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:12.667374 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447802ad-8235-408d-a6bf-660e2ff89cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:12.668582 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447802ad-8235-408d-a6bf-660e2ff89cb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:12.668582 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447802ad-8235-408d-a6bf-660e2ff89cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:12.668893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447802ad-8235-408d-a6bf-660e2ff89cb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:12.669345 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447802ad-8235-408d-a6bf-660e2ff89cb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:12.671512 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447802ad-8235-408d-a6bf-660e2ff89cb8 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:12.732053 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 998/1983] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:12:12 2022] POST /identity/v3/users => generated 295 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:12:12.736311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242c682-0f93-456d-aa07-1aca5c302d4c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:12.751720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242c682-0f93-456d-aa07-1aca5c302d4c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:12.759705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a563e5-384d-4631-995e-1f0e7cc392b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:12.760604 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a563e5-384d-4631-995e-1f0e7cc392b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:12.760794 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a563e5-384d-4631-995e-1f0e7cc392b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:12.761088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a563e5-384d-4631-995e-1f0e7cc392b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:12.761826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a563e5-384d-4631-995e-1f0e7cc392b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:12.763818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a563e5-384d-4631-995e-1f0e7cc392b4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:12.790255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 986/1984] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:12 2022] GET /identity/v3/roles => generated 1427 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:12.794818 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447802ad-8235-408d-a6bf-660e2ff89cb8 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:12.806637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447802ad-8235-408d-a6bf-660e2ff89cb8 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:12.818595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f311306-e148-4f1f-b64e-af2e0b8f53dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:12.819381 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f311306-e148-4f1f-b64e-af2e0b8f53dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:12.819728 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f311306-e148-4f1f-b64e-af2e0b8f53dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:12.820078 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f311306-e148-4f1f-b64e-af2e0b8f53dd admin admin] PATH_INFO: `/v3/projects/f41d5cf61d8645439d0385db0723fb89/users/e2d54d5c32e24d67b97ec7a5724c9c75/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:12.830597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7420b1b-17f6-4e33-9d51-212a1ffd6d8a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:12.830894 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7420b1b-17f6-4e33-9d51-212a1ffd6d8a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:12.831177 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7420b1b-17f6-4e33-9d51-212a1ffd6d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:12.858651 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f311306-e148-4f1f-b64e-af2e0b8f53dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f41d5cf61d8645439d0385db0723fb89, user_id=e2d54d5c32e24d67b97ec7a5724c9c75, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:12.863264 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c7420b1b-17f6-4e33-9d51-212a1ffd6d8a None None] MFA Rules not processed for user `f08fe9a161dd4757a70d4c2075436901`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:12.863957 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c7420b1b-17f6-4e33-9d51-212a1ffd6d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:12.866638 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f311306-e148-4f1f-b64e-af2e0b8f53dd admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:12.885151 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 999/1985] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:12:12 2022] PUT /identity/v3/projects/f41d5cf61d8645439d0385db0723fb89/users/e2d54d5c32e24d67b97ec7a5724c9c75/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:12:12.889633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f311306-e148-4f1f-b64e-af2e0b8f53dd admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:12.938678 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:12.967862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f311306-e148-4f1f-b64e-af2e0b8f53dd admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:12.969510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 987/1986] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:12.974078 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a19d7-dbde-4081-9ca7-3e7db2810832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:12.975433 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a19d7-dbde-4081-9ca7-3e7db2810832 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:12.975725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a19d7-dbde-4081-9ca7-3e7db2810832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:12.976032 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a19d7-dbde-4081-9ca7-3e7db2810832 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:12.977410 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a19d7-dbde-4081-9ca7-3e7db2810832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:12.978511 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a19d7-dbde-4081-9ca7-3e7db2810832 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:12.994054 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1000/1987] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:12 2022] GET /identity/v3/roles => generated 1427 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:12.995356 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7420b1b-17f6-4e33-9d51-212a1ffd6d8a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:13.008563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7420b1b-17f6-4e33-9d51-212a1ffd6d8a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:13.013968 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f801d-4f89-4c6d-8319-e0ea2dcfd556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:13.015422 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f801d-4f89-4c6d-8319-e0ea2dcfd556 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:13.015713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f801d-4f89-4c6d-8319-e0ea2dcfd556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:13.018060 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f801d-4f89-4c6d-8319-e0ea2dcfd556 admin admin] PATH_INFO: `/v3/projects/f41d5cf61d8645439d0385db0723fb89/users/e2d54d5c32e24d67b97ec7a5724c9c75/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:13.019333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f801d-4f89-4c6d-8319-e0ea2dcfd556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f41d5cf61d8645439d0385db0723fb89, user_id=e2d54d5c32e24d67b97ec7a5724c9c75, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:13.023500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f801d-4f89-4c6d-8319-e0ea2dcfd556 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:13.045906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 988/1988] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:12:12 2022] PUT /identity/v3/projects/f41d5cf61d8645439d0385db0723fb89/users/e2d54d5c32e24d67b97ec7a5724c9c75/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:12:13.049778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349f57a-cf6f-4c35-95a5-bf8245c8bbbc None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:13.050200 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349f57a-cf6f-4c35-95a5-bf8245c8bbbc None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:13.050656 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349f57a-cf6f-4c35-95a5-bf8245c8bbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:13.085352 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-9349f57a-cf6f-4c35-95a5-bf8245c8bbbc None None] MFA Rules not processed for user `e2d54d5c32e24d67b97ec7a5724c9c75`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:13.086799 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-9349f57a-cf6f-4c35-95a5-bf8245c8bbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:13.191726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1001/1989] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:13.205409 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f801d-4f89-4c6d-8319-e0ea2dcfd556 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:13.207301 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e91f801d-4f89-4c6d-8319-e0ea2dcfd556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:13.296305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f801d-4f89-4c6d-8319-e0ea2dcfd556 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:13.300434 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662c612-47c7-4851-9fcb-8298169c3174 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:13.301203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662c612-47c7-4851-9fcb-8298169c3174 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:13.301487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662c612-47c7-4851-9fcb-8298169c3174 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:13.301765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662c612-47c7-4851-9fcb-8298169c3174 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:13.386454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662c612-47c7-4851-9fcb-8298169c3174 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:13.387911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662c612-47c7-4851-9fcb-8298169c3174 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:13.413209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 989/1990] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:12:20.244822 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9349f57a-cf6f-4c35-95a5-bf8245c8bbbc None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:20.249511 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:20.257019 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9349f57a-cf6f-4c35-95a5-bf8245c8bbbc None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:20.262860 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236b089-39bc-4ee9-be01-3d1abd120f19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:20.263485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236b089-39bc-4ee9-be01-3d1abd120f19 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:20.263764 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236b089-39bc-4ee9-be01-3d1abd120f19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:20.264037 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236b089-39bc-4ee9-be01-3d1abd120f19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:20.276700 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0236b089-39bc-4ee9-be01-3d1abd120f19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:20.278202 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0236b089-39bc-4ee9-be01-3d1abd120f19 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:20.290486 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1002/1991] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:12:23.394547 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=56,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:25.936265 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662c612-47c7-4851-9fcb-8298169c3174 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:25.937825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c662c612-47c7-4851-9fcb-8298169c3174 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:26.044520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662c612-47c7-4851-9fcb-8298169c3174 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:26.050188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ea35c-3dc8-4f5b-831a-7b039bd7fbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:26.050869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ea35c-3dc8-4f5b-831a-7b039bd7fbd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:26.051156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ea35c-3dc8-4f5b-831a-7b039bd7fbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:26.051438 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ea35c-3dc8-4f5b-831a-7b039bd7fbd0 admin admin] PATH_INFO: `/v3/users/2e1b302883cf4786b0369f911bb28228` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:26.052846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ea35c-3dc8-4f5b-831a-7b039bd7fbd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e1b302883cf4786b0369f911bb28228)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:26.054901 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ea35c-3dc8-4f5b-831a-7b039bd7fbd0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:26.129696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-092ea35c-3dc8-4f5b-831a-7b039bd7fbd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e1b302883cf4786b0369f911bb28228'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:26.140348 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-092ea35c-3dc8-4f5b-831a-7b039bd7fbd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e1b302883cf4786b0369f911bb28228'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:26.141149 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-092ea35c-3dc8-4f5b-831a-7b039bd7fbd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e1b302883cf4786b0369f911bb28228'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:26.163229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-092ea35c-3dc8-4f5b-831a-7b039bd7fbd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e1b302883cf4786b0369f911bb28228'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:26.164026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-092ea35c-3dc8-4f5b-831a-7b039bd7fbd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e1b302883cf4786b0369f911bb28228'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:26.186214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 990/1992] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:12:25 2022] DELETE /identity/v3/users/2e1b302883cf4786b0369f911bb28228 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:26.191310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cf72c-d7ff-42a1-99f5-879c1534f00d None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:26.192256 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cf72c-d7ff-42a1-99f5-879c1534f00d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:26.192535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cf72c-d7ff-42a1-99f5-879c1534f00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:26.212663 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-e15cf72c-d7ff-42a1-99f5-879c1534f00d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:26.214227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e15cf72c-d7ff-42a1-99f5-879c1534f00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:26.313758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1003/1993] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:12:26.323680 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ea35c-3dc8-4f5b-831a-7b039bd7fbd0 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:26.325176 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-092ea35c-3dc8-4f5b-831a-7b039bd7fbd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:26.400170 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ea35c-3dc8-4f5b-831a-7b039bd7fbd0 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:26.404489 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffa378-3620-4fc4-9e74-c288c64ad44e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:26.405147 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffa378-3620-4fc4-9e74-c288c64ad44e service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:26.405434 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffa378-3620-4fc4-9e74-c288c64ad44e service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:26.405708 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffa378-3620-4fc4-9e74-c288c64ad44e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:26.415585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffa378-3620-4fc4-9e74-c288c64ad44e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:26.416933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffa378-3620-4fc4-9e74-c288c64ad44e service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:26.427580 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 991/1994] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:26 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:12:26.726777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15cf72c-d7ff-42a1-99f5-879c1534f00d None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:26.728290 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e15cf72c-d7ff-42a1-99f5-879c1534f00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:26.746531 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15cf72c-d7ff-42a1-99f5-879c1534f00d None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:26.752842 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4e613-f4b2-4503-bfa6-ea91f3252fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:26.753977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4e613-f4b2-4503-bfa6-ea91f3252fa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:26.754391 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4e613-f4b2-4503-bfa6-ea91f3252fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:26.754705 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4e613-f4b2-4503-bfa6-ea91f3252fa0 admin admin] PATH_INFO: `/v3/projects/a1cc8acac44d4625b074d2793d97c120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:26.756210 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4e613-f4b2-4503-bfa6-ea91f3252fa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1cc8acac44d4625b074d2793d97c120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:26.759376 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4e613-f4b2-4503-bfa6-ea91f3252fa0 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:26.838235 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dda4e613-f4b2-4503-bfa6-ea91f3252fa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1cc8acac44d4625b074d2793d97c120', 'request_id': 'req-dda4e613-f4b2-4503-bfa6-ea91f3252fa0'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:26.848228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dda4e613-f4b2-4503-bfa6-ea91f3252fa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1cc8acac44d4625b074d2793d97c120', 'request_id': 'req-dda4e613-f4b2-4503-bfa6-ea91f3252fa0'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:26.863252 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dda4e613-f4b2-4503-bfa6-ea91f3252fa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1cc8acac44d4625b074d2793d97c120', 'request_id': 'req-dda4e613-f4b2-4503-bfa6-ea91f3252fa0'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:26.879047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dda4e613-f4b2-4503-bfa6-ea91f3252fa0 admin admin] The token cache is being invalidate because project a1cc8acac44d4625b074d2793d97c120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:12:26.879350 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dda4e613-f4b2-4503-bfa6-ea91f3252fa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:26.880423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dda4e613-f4b2-4503-bfa6-ea91f3252fa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:26.881950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1004/1995] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:12:26 2022] DELETE /identity/v3/projects/a1cc8acac44d4625b074d2793d97c120 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:26.887086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a15fe2-d52c-4d04-aa8f-0df9677d8d48 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:26.887636 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a15fe2-d52c-4d04-aa8f-0df9677d8d48 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:26.887998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a15fe2-d52c-4d04-aa8f-0df9677d8d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:26.910225 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-38a15fe2-d52c-4d04-aa8f-0df9677d8d48 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:26.914708 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-38a15fe2-d52c-4d04-aa8f-0df9677d8d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:26.932772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:26.979073 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 992/1996] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:26.987245 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d9ee1-1a3a-4f13-8e9d-e8e72b3a9970 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:26.987537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d9ee1-1a3a-4f13-8e9d-e8e72b3a9970 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:26.987819 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d9ee1-1a3a-4f13-8e9d-e8e72b3a9970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:27.007825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-436d9ee1-1a3a-4f13-8e9d-e8e72b3a9970 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:27.009060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-436d9ee1-1a3a-4f13-8e9d-e8e72b3a9970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:27.017649 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1005/1997] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:27.022137 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a15fe2-d52c-4d04-aa8f-0df9677d8d48 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:27.033154 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a15fe2-d52c-4d04-aa8f-0df9677d8d48 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:27.039777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48730e5c-57d7-4930-845e-e2bcb3baf998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:27.040433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48730e5c-57d7-4930-845e-e2bcb3baf998 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:27.040699 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48730e5c-57d7-4930-845e-e2bcb3baf998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:27.040972 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48730e5c-57d7-4930-845e-e2bcb3baf998 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:27.041766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48730e5c-57d7-4930-845e-e2bcb3baf998 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:12:27.042062 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48730e5c-57d7-4930-845e-e2bcb3baf998 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:27.043694 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48730e5c-57d7-4930-845e-e2bcb3baf998 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:27.056338 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 993/1998] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:12:27 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:12:27.060762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d9ee1-1a3a-4f13-8e9d-e8e72b3a9970 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:27.072470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d9ee1-1a3a-4f13-8e9d-e8e72b3a9970 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:27.084856 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3de94-aeec-40cb-872d-71da10f4eace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:27.084856 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3de94-aeec-40cb-872d-71da10f4eace admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:27.084856 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3de94-aeec-40cb-872d-71da10f4eace admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:27.084856 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3de94-aeec-40cb-872d-71da10f4eace admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:27.085248 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3de94-aeec-40cb-872d-71da10f4eace admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:27.085248 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3de94-aeec-40cb-872d-71da10f4eace admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:27.114453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1006/1999] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:12:27 2022] POST /identity/v3/projects => generated 360 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:12:27.119423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48730e5c-57d7-4930-845e-e2bcb3baf998 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:27.130122 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48730e5c-57d7-4930-845e-e2bcb3baf998 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:27.221214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47c2ad-cd3e-4285-a07c-acb0008cc000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:27.221847 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47c2ad-cd3e-4285-a07c-acb0008cc000 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:27.222131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47c2ad-cd3e-4285-a07c-acb0008cc000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:27.222406 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47c2ad-cd3e-4285-a07c-acb0008cc000 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:27.223071 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47c2ad-cd3e-4285-a07c-acb0008cc000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:27.224979 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47c2ad-cd3e-4285-a07c-acb0008cc000 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:27.249360 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 994/2000] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:12:27 2022] POST /identity/v3/users => generated 281 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:12:27.254762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3de94-aeec-40cb-872d-71da10f4eace admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:27.265822 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3de94-aeec-40cb-872d-71da10f4eace admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:27.272272 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95df14-db78-4814-878c-821b65716c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:27.273215 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95df14-db78-4814-878c-821b65716c53 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:27.273577 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95df14-db78-4814-878c-821b65716c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:27.273927 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95df14-db78-4814-878c-821b65716c53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:27.274712 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c95df14-db78-4814-878c-821b65716c53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:27.276664 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c95df14-db78-4814-878c-821b65716c53 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:27.297183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1007/2001] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:27 2022] GET /identity/v3/roles => generated 1427 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:27.301701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47c2ad-cd3e-4285-a07c-acb0008cc000 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:27.331482 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47c2ad-cd3e-4285-a07c-acb0008cc000 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:27.348640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7ad5b-5420-4cbc-8aad-3040a801326f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:27.349362 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7ad5b-5420-4cbc-8aad-3040a801326f admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:27.349667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7ad5b-5420-4cbc-8aad-3040a801326f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:27.351074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7ad5b-5420-4cbc-8aad-3040a801326f admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:27.351074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7ad5b-5420-4cbc-8aad-3040a801326f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:27.352263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7ad5b-5420-4cbc-8aad-3040a801326f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:27.384160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 995/2002] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:27 2022] GET /identity/v3/roles => generated 1427 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:27.401611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95df14-db78-4814-878c-821b65716c53 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:27.428082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95df14-db78-4814-878c-821b65716c53 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:27.434195 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c5d1a-8ac3-4b39-87c8-501c836e9800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:27.434903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c5d1a-8ac3-4b39-87c8-501c836e9800 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:27.435185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c5d1a-8ac3-4b39-87c8-501c836e9800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:27.435466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c5d1a-8ac3-4b39-87c8-501c836e9800 admin admin] PATH_INFO: `/v3/projects/03362621ea2b48258e6afde836f03da8/users/176856577515427b87e18a407108add1/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:27.457396 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c5d1a-8ac3-4b39-87c8-501c836e9800 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03362621ea2b48258e6afde836f03da8, user_id=176856577515427b87e18a407108add1, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:27.460673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c5d1a-8ac3-4b39-87c8-501c836e9800 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:27.480015 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1008/2003] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:12:27 2022] PUT /identity/v3/projects/03362621ea2b48258e6afde836f03da8/users/176856577515427b87e18a407108add1/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:27.508302 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6a7e2-914a-4cab-b138-b919770a79ff None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:27.508302 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6a7e2-914a-4cab-b138-b919770a79ff None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:27.508302 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6a7e2-914a-4cab-b138-b919770a79ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:27.543791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-85e6a7e2-914a-4cab-b138-b919770a79ff None None] MFA Rules not processed for user `176856577515427b87e18a407108add1`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:27.545097 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-85e6a7e2-914a-4cab-b138-b919770a79ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:27.624353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 996/2004] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:27.664195 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c5d1a-8ac3-4b39-87c8-501c836e9800 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:27.666055 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d26c5d1a-8ac3-4b39-87c8-501c836e9800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:27.681147 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:27.762853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c5d1a-8ac3-4b39-87c8-501c836e9800 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:27.767525 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c54cb-e21c-4662-9161-134127cc8dd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:27.768166 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c54cb-e21c-4662-9161-134127cc8dd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:27.768452 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c54cb-e21c-4662-9161-134127cc8dd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:27.768730 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c54cb-e21c-4662-9161-134127cc8dd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:27.788493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c54cb-e21c-4662-9161-134127cc8dd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:27.790369 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c54cb-e21c-4662-9161-134127cc8dd4 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:27.803924 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1009/2005] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:12:27.820044 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6a7e2-914a-4cab-b138-b919770a79ff None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:27.846396 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:27.903659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6a7e2-914a-4cab-b138-b919770a79ff None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:27.909869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0bb3b-ca42-442e-8800-115be1d7fb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:27.910627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0bb3b-ca42-442e-8800-115be1d7fb3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:27.911116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0bb3b-ca42-442e-8800-115be1d7fb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:27.911278 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0bb3b-ca42-442e-8800-115be1d7fb3b admin admin] PATH_INFO: `/v3/users/176856577515427b87e18a407108add1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:27.913377 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0bb3b-ca42-442e-8800-115be1d7fb3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=176856577515427b87e18a407108add1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:27.914957 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0bb3b-ca42-442e-8800-115be1d7fb3b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:27.978664 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5dd0bb3b-ca42-442e-8800-115be1d7fb3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '176856577515427b87e18a407108add1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:27.986504 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5dd0bb3b-ca42-442e-8800-115be1d7fb3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '176856577515427b87e18a407108add1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:27.987220 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5dd0bb3b-ca42-442e-8800-115be1d7fb3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '176856577515427b87e18a407108add1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:28.002731 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5dd0bb3b-ca42-442e-8800-115be1d7fb3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '176856577515427b87e18a407108add1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:28.003424 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5dd0bb3b-ca42-442e-8800-115be1d7fb3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '176856577515427b87e18a407108add1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:28.023659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 997/2006] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:12:27 2022] DELETE /identity/v3/users/176856577515427b87e18a407108add1 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:28.028353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085e629-f748-44b1-a134-49984775d935 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:28.028649 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085e629-f748-44b1-a134-49984775d935 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:28.028930 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085e629-f748-44b1-a134-49984775d935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:28.060507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-8085e629-f748-44b1-a134-49984775d935 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:28.061868 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-8085e629-f748-44b1-a134-49984775d935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:28.149359 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1010/2007] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:28.168778 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0bb3b-ca42-442e-8800-115be1d7fb3b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:28.170948 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5dd0bb3b-ca42-442e-8800-115be1d7fb3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:28.239536 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0bb3b-ca42-442e-8800-115be1d7fb3b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:28.243280 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c742a-67c1-4109-86bb-74e337ec4429 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:28.243907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c742a-67c1-4109-86bb-74e337ec4429 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:28.244193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c742a-67c1-4109-86bb-74e337ec4429 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:28.244467 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c742a-67c1-4109-86bb-74e337ec4429 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:28.256692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c742a-67c1-4109-86bb-74e337ec4429 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:28.256878 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c742a-67c1-4109-86bb-74e337ec4429 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:28.268049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 998/2008] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:12:28.704627 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085e629-f748-44b1-a134-49984775d935 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:28.707873 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-8085e629-f748-44b1-a134-49984775d935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:28.722742 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085e629-f748-44b1-a134-49984775d935 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:28.729416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56889fa8-f351-498b-8aea-7c4b162a11db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:28.730288 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56889fa8-f351-498b-8aea-7c4b162a11db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:28.730674 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56889fa8-f351-498b-8aea-7c4b162a11db admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:28.731051 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56889fa8-f351-498b-8aea-7c4b162a11db admin admin] PATH_INFO: `/v3/projects/03362621ea2b48258e6afde836f03da8` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:28.732547 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56889fa8-f351-498b-8aea-7c4b162a11db admin admin] RBAC: Authorizing `identity:delete_project(project_id=03362621ea2b48258e6afde836f03da8)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:28.743325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56889fa8-f351-498b-8aea-7c4b162a11db admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:28.818550 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-56889fa8-f351-498b-8aea-7c4b162a11db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03362621ea2b48258e6afde836f03da8', 'request_id': 'req-56889fa8-f351-498b-8aea-7c4b162a11db'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:28.827617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-56889fa8-f351-498b-8aea-7c4b162a11db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03362621ea2b48258e6afde836f03da8', 'request_id': 'req-56889fa8-f351-498b-8aea-7c4b162a11db'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:28.836919 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-56889fa8-f351-498b-8aea-7c4b162a11db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03362621ea2b48258e6afde836f03da8', 'request_id': 'req-56889fa8-f351-498b-8aea-7c4b162a11db'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:28.847847 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-56889fa8-f351-498b-8aea-7c4b162a11db admin admin] The token cache is being invalidate because project 03362621ea2b48258e6afde836f03da8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:12:28.848183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-56889fa8-f351-498b-8aea-7c4b162a11db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:28.848817 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-56889fa8-f351-498b-8aea-7c4b162a11db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:28.850386 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1011/2009] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:12:28 2022] DELETE /identity/v3/projects/03362621ea2b48258e6afde836f03da8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:28.855376 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e86ec0-5832-43e0-bd05-bf3b80068021 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:28.855668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e86ec0-5832-43e0-bd05-bf3b80068021 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:28.855951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e86ec0-5832-43e0-bd05-bf3b80068021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:28.876713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-21e86ec0-5832-43e0-bd05-bf3b80068021 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:28.877960 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-21e86ec0-5832-43e0-bd05-bf3b80068021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:28.926231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:28.956425 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 999/2010] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:28.965592 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414b131-a986-4b77-a465-7b70b4ca85de None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:28.965898 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414b131-a986-4b77-a465-7b70b4ca85de None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:28.966175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414b131-a986-4b77-a465-7b70b4ca85de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:28.987663 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-8414b131-a986-4b77-a465-7b70b4ca85de None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:28.988881 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-8414b131-a986-4b77-a465-7b70b4ca85de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:28.997073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1012/2011] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:29.002576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e86ec0-5832-43e0-bd05-bf3b80068021 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:29.014243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e86ec0-5832-43e0-bd05-bf3b80068021 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:29.020299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d6b4f-be73-4213-bb9a-37229f4e15af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:29.020953 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d6b4f-be73-4213-bb9a-37229f4e15af admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:29.021279 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d6b4f-be73-4213-bb9a-37229f4e15af admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:29.021570 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d6b4f-be73-4213-bb9a-37229f4e15af admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:29.022294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d6b4f-be73-4213-bb9a-37229f4e15af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:12:29.022604 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d6b4f-be73-4213-bb9a-37229f4e15af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:29.024235 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d6b4f-be73-4213-bb9a-37229f4e15af admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:29.037937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1000/2012] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:12:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:12:29.042323 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414b131-a986-4b77-a465-7b70b4ca85de None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:29.056358 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414b131-a986-4b77-a465-7b70b4ca85de None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:29.064249 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fcfa4b-9a91-47b4-a2c4-bff9e8805321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:29.064898 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcfa4b-9a91-47b4-a2c4-bff9e8805321 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:29.065208 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcfa4b-9a91-47b4-a2c4-bff9e8805321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:29.065485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcfa4b-9a91-47b4-a2c4-bff9e8805321 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:29.066162 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fcfa4b-9a91-47b4-a2c4-bff9e8805321 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:29.068096 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fcfa4b-9a91-47b4-a2c4-bff9e8805321 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:29.100999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1013/2013] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:12:29 2022] POST /identity/v3/projects => generated 372 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:12:29.105206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d6b4f-be73-4213-bb9a-37229f4e15af admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:29.120881 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d6b4f-be73-4213-bb9a-37229f4e15af admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:29.193338 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2991cf7-3637-4f0a-8623-b74558b91ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:29.193982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2991cf7-3637-4f0a-8623-b74558b91ba9 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:29.194266 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2991cf7-3637-4f0a-8623-b74558b91ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:29.194551 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2991cf7-3637-4f0a-8623-b74558b91ba9 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:29.195265 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2991cf7-3637-4f0a-8623-b74558b91ba9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:29.197279 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2991cf7-3637-4f0a-8623-b74558b91ba9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:29.223234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1001/2014] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:12:29 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:12:29.227962 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fcfa4b-9a91-47b4-a2c4-bff9e8805321 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:29.241985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fcfa4b-9a91-47b4-a2c4-bff9e8805321 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:29.248626 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4e744-319d-4729-88ad-0cb34e1d2d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:29.249608 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4e744-319d-4729-88ad-0cb34e1d2d2f admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:29.249894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4e744-319d-4729-88ad-0cb34e1d2d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:29.250171 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4e744-319d-4729-88ad-0cb34e1d2d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:29.252071 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4e744-319d-4729-88ad-0cb34e1d2d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:29.253764 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4e744-319d-4729-88ad-0cb34e1d2d2f admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:29.255512 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:29.269502 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1014/2015] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:29 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:29.271962 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2991cf7-3637-4f0a-8623-b74558b91ba9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:29.282795 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2991cf7-3637-4f0a-8623-b74558b91ba9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:29.287890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e62e0c-bb8f-41e5-b0be-3a8f70084fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:29.288551 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e62e0c-bb8f-41e5-b0be-3a8f70084fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:29.288833 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e62e0c-bb8f-41e5-b0be-3a8f70084fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:29.289138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e62e0c-bb8f-41e5-b0be-3a8f70084fd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:29.290226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e62e0c-bb8f-41e5-b0be-3a8f70084fd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:29.291828 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e62e0c-bb8f-41e5-b0be-3a8f70084fd0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:29.303848 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1002/2016] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:29 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:29.308214 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4e744-319d-4729-88ad-0cb34e1d2d2f admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:29.319949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4e744-319d-4729-88ad-0cb34e1d2d2f admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:29.324668 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b595a-87ea-4fe3-9aa3-5a1d39b86938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:29.325394 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b595a-87ea-4fe3-9aa3-5a1d39b86938 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:29.325664 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b595a-87ea-4fe3-9aa3-5a1d39b86938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:29.325942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b595a-87ea-4fe3-9aa3-5a1d39b86938 admin admin] PATH_INFO: `/v3/projects/d1d95a5c8c834b9f8f1e20e9990b54df/users/fd776c3e90a94dcf94faa55235e9cb85/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:29.342448 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b595a-87ea-4fe3-9aa3-5a1d39b86938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d95a5c8c834b9f8f1e20e9990b54df, user_id=fd776c3e90a94dcf94faa55235e9cb85, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:29.345764 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b595a-87ea-4fe3-9aa3-5a1d39b86938 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:29.369615 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1015/2017] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:12:29 2022] PUT /identity/v3/projects/d1d95a5c8c834b9f8f1e20e9990b54df/users/fd776c3e90a94dcf94faa55235e9cb85/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:29.382022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611739c-740b-4a41-8363-5c4242a8793c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:29.382344 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611739c-740b-4a41-8363-5c4242a8793c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:29.382607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611739c-740b-4a41-8363-5c4242a8793c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:29.411480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d611739c-740b-4a41-8363-5c4242a8793c None None] MFA Rules not processed for user `fd776c3e90a94dcf94faa55235e9cb85`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:29.412766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d611739c-740b-4a41-8363-5c4242a8793c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:29.482276 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1003/2018] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:29.494672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b595a-87ea-4fe3-9aa3-5a1d39b86938 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:29.496382 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f66b595a-87ea-4fe3-9aa3-5a1d39b86938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:29.556694 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b595a-87ea-4fe3-9aa3-5a1d39b86938 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:29.560398 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83561a-2362-4754-9c7d-9fbecd0edaf7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd959ddd087724547a11c3d4c796d8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:29.562869 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83561a-2362-4754-9c7d-9fbecd0edaf7 service glance] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:29.562869 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83561a-2362-4754-9c7d-9fbecd0edaf7 service glance] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:29.562869 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83561a-2362-4754-9c7d-9fbecd0edaf7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:29.570838 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83561a-2362-4754-9c7d-9fbecd0edaf7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:29.572279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83561a-2362-4754-9c7d-9fbecd0edaf7 service glance] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:29.581788 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1016/2019] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:12:39.479330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:39.577451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=38,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:43.170398 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbde613-ec54-45d1-a92a-d3636cbb6223 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:43.170398 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbde613-ec54-45d1-a92a-d3636cbb6223 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:43.170932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbde613-ec54-45d1-a92a-d3636cbb6223 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:43.172846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1004/2020] 10.222.0.100 () {54 vars in 926 bytes} [Tue Aug 2 09:12:43 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:12:43.184044 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f958e-9b4d-42fb-93c2-6ddb13fde729 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:43.184601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f958e-9b4d-42fb-93c2-6ddb13fde729 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:43.184601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f958e-9b4d-42fb-93c2-6ddb13fde729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:43.207393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-4e6f958e-9b4d-42fb-93c2-6ddb13fde729 None None] MFA Rules not processed for user `b40981fe179a43dc81553d3594dad649`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:43.209316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-4e6f958e-9b4d-42fb-93c2-6ddb13fde729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:43.250981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbde613-ec54-45d1-a92a-d3636cbb6223 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:43.254085 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fdbde613-ec54-45d1-a92a-d3636cbb6223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:43.293745 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1017/2021] 10.222.0.100 () {58 vars in 1023 bytes} [Tue Aug 2 09:12:43 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:43.307823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f958e-9b4d-42fb-93c2-6ddb13fde729 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:43.309646 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-4e6f958e-9b4d-42fb-93c2-6ddb13fde729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:43.324429 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f958e-9b4d-42fb-93c2-6ddb13fde729 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:43.330482 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471877fc-a1ac-4e95-96aa-5178c8b14d6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b40981fe179a43dc81553d3594dad649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:43.331194 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471877fc-a1ac-4e95-96aa-5178c8b14d6f service placement] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:43.331550 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471877fc-a1ac-4e95-96aa-5178c8b14d6f service placement] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:43.331905 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471877fc-a1ac-4e95-96aa-5178c8b14d6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:43.335315 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbde613-ec54-45d1-a92a-d3636cbb6223 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:43.341191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471877fc-a1ac-4e95-96aa-5178c8b14d6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:43.342886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471877fc-a1ac-4e95-96aa-5178c8b14d6f service placement] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:43.343890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5deeb6-f481-418b-b532-959eca7b5eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:43.345061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5deeb6-f481-418b-b532-959eca7b5eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:43.345522 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5deeb6-f481-418b-b532-959eca7b5eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:43.345862 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5deeb6-f481-418b-b532-959eca7b5eef admin admin] PATH_INFO: `/v3/users/c2782f4783624ca39ace71143b3b9237` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:43.347122 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5deeb6-f481-418b-b532-959eca7b5eef admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2782f4783624ca39ace71143b3b9237)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:43.349486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5deeb6-f481-418b-b532-959eca7b5eef admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:43.360227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1018/2022] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:43 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:12:43.414536 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7d5deeb6-f481-418b-b532-959eca7b5eef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2782f4783624ca39ace71143b3b9237'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:43.423070 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7d5deeb6-f481-418b-b532-959eca7b5eef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2782f4783624ca39ace71143b3b9237'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:43.425222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7d5deeb6-f481-418b-b532-959eca7b5eef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2782f4783624ca39ace71143b3b9237'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:43.432774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7d5deeb6-f481-418b-b532-959eca7b5eef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2782f4783624ca39ace71143b3b9237'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:43.433514 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7d5deeb6-f481-418b-b532-959eca7b5eef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2782f4783624ca39ace71143b3b9237'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:43.454949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1005/2023] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:12:43 2022] DELETE /identity/v3/users/c2782f4783624ca39ace71143b3b9237 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:43.682187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471877fc-a1ac-4e95-96aa-5178c8b14d6f service placement] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:43.693957 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471877fc-a1ac-4e95-96aa-5178c8b14d6f service placement] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:43.758604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c182f91-7f93-49e1-88f7-b550718036ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:43.759387 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c182f91-7f93-49e1-88f7-b550718036ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:43.759616 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c182f91-7f93-49e1-88f7-b550718036ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:43.759876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c182f91-7f93-49e1-88f7-b550718036ca admin admin] PATH_INFO: `/v3/projects/cde77e494bb54329a58883c255a8c2c0` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:43.761362 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c182f91-7f93-49e1-88f7-b550718036ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=cde77e494bb54329a58883c255a8c2c0)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:43.763939 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c182f91-7f93-49e1-88f7-b550718036ca admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:43.834997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5c182f91-7f93-49e1-88f7-b550718036ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cde77e494bb54329a58883c255a8c2c0', 'request_id': 'req-5c182f91-7f93-49e1-88f7-b550718036ca'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:43.842986 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5c182f91-7f93-49e1-88f7-b550718036ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cde77e494bb54329a58883c255a8c2c0', 'request_id': 'req-5c182f91-7f93-49e1-88f7-b550718036ca'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:43.852963 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5c182f91-7f93-49e1-88f7-b550718036ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cde77e494bb54329a58883c255a8c2c0', 'request_id': 'req-5c182f91-7f93-49e1-88f7-b550718036ca'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:43.868177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5c182f91-7f93-49e1-88f7-b550718036ca admin admin] The token cache is being invalidate because project cde77e494bb54329a58883c255a8c2c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:12:43.869647 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5c182f91-7f93-49e1-88f7-b550718036ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:43.870287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5c182f91-7f93-49e1-88f7-b550718036ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:43.873081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1019/2024] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:12:43 2022] DELETE /identity/v3/projects/cde77e494bb54329a58883c255a8c2c0 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:43.879026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58613a1c-4194-4397-8427-5d17b64bdf52 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:43.879316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58613a1c-4194-4397-8427-5d17b64bdf52 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:43.879643 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58613a1c-4194-4397-8427-5d17b64bdf52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:43.926537 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-58613a1c-4194-4397-8427-5d17b64bdf52 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:43.928594 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-58613a1c-4194-4397-8427-5d17b64bdf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:44.008177 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1006/2025] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:44.016809 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a79ffc-912b-43e1-a33f-f4ce553f12db None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:44.017051 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a79ffc-912b-43e1-a33f-f4ce553f12db None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:44.017406 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a79ffc-912b-43e1-a33f-f4ce553f12db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:44.049551 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-b7a79ffc-912b-43e1-a33f-f4ce553f12db None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:44.050825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b7a79ffc-912b-43e1-a33f-f4ce553f12db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:44.059503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1020/2026] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:12:44.066520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58613a1c-4194-4397-8427-5d17b64bdf52 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:44.077095 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58613a1c-4194-4397-8427-5d17b64bdf52 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:44.091081 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3e9a7-ace0-453e-a769-3efcf5f5aeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:44.091734 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3e9a7-ace0-453e-a769-3efcf5f5aeb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:44.097047 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3e9a7-ace0-453e-a769-3efcf5f5aeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:44.097047 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3e9a7-ace0-453e-a769-3efcf5f5aeb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:44.097047 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3e9a7-ace0-453e-a769-3efcf5f5aeb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:12:44.097047 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3e9a7-ace0-453e-a769-3efcf5f5aeb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:44.097047 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3e9a7-ace0-453e-a769-3efcf5f5aeb6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:44.119365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1007/2027] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:12:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:12:44.125646 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a79ffc-912b-43e1-a33f-f4ce553f12db None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:44.135632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a79ffc-912b-43e1-a33f-f4ce553f12db None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:44.141092 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801fde8-74ca-4b80-9dfd-eba104879a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:44.142831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801fde8-74ca-4b80-9dfd-eba104879a25 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:44.143095 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801fde8-74ca-4b80-9dfd-eba104879a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:44.143374 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801fde8-74ca-4b80-9dfd-eba104879a25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:44.144076 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801fde8-74ca-4b80-9dfd-eba104879a25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:44.146069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801fde8-74ca-4b80-9dfd-eba104879a25 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:44.151144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:44.187036 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1021/2028] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:12:44 2022] POST /identity/v3/projects => generated 376 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:12:44.195133 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3e9a7-ace0-453e-a769-3efcf5f5aeb6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:44.212190 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3e9a7-ace0-453e-a769-3efcf5f5aeb6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:44.218144 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:44.305306 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a46831-dacb-485f-a949-0332ae4a9edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:44.305863 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a46831-dacb-485f-a949-0332ae4a9edf admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:44.306149 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a46831-dacb-485f-a949-0332ae4a9edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:44.306430 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a46831-dacb-485f-a949-0332ae4a9edf admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:44.307705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a46831-dacb-485f-a949-0332ae4a9edf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:44.309679 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a46831-dacb-485f-a949-0332ae4a9edf admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:44.334542 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1008/2029] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:12:44 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:12:44.338700 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801fde8-74ca-4b80-9dfd-eba104879a25 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:44.350814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801fde8-74ca-4b80-9dfd-eba104879a25 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:44.360909 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a274ca3-751e-4e56-b640-bf8f3f3d1798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:44.361604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a274ca3-751e-4e56-b640-bf8f3f3d1798 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:44.361968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a274ca3-751e-4e56-b640-bf8f3f3d1798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:44.362289 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a274ca3-751e-4e56-b640-bf8f3f3d1798 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:44.362995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a274ca3-751e-4e56-b640-bf8f3f3d1798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:44.364644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a274ca3-751e-4e56-b640-bf8f3f3d1798 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:44.376888 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1022/2030] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:44 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:44.381505 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a46831-dacb-485f-a949-0332ae4a9edf admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:44.394948 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a46831-dacb-485f-a949-0332ae4a9edf admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:44.399987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cccfa0-2745-4b09-b40e-c812797d4713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:44.400668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cccfa0-2745-4b09-b40e-c812797d4713 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:44.400950 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cccfa0-2745-4b09-b40e-c812797d4713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:44.401272 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cccfa0-2745-4b09-b40e-c812797d4713 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:44.401982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cccfa0-2745-4b09-b40e-c812797d4713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:44.403765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cccfa0-2745-4b09-b40e-c812797d4713 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:44.424480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1009/2031] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:44 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:44.427317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a274ca3-751e-4e56-b640-bf8f3f3d1798 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:44.441219 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a274ca3-751e-4e56-b640-bf8f3f3d1798 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:44.446154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97f80f-abeb-4ffc-8465-59ac413bc643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:44.447062 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97f80f-abeb-4ffc-8465-59ac413bc643 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:44.447360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97f80f-abeb-4ffc-8465-59ac413bc643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:44.447803 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97f80f-abeb-4ffc-8465-59ac413bc643 admin admin] PATH_INFO: `/v3/projects/e2757dae0d3a45a29427faed00e98996/users/18728d1c3451497480556461b4ec80d5/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:44.471548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97f80f-abeb-4ffc-8465-59ac413bc643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2757dae0d3a45a29427faed00e98996, user_id=18728d1c3451497480556461b4ec80d5, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:44.474457 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97f80f-abeb-4ffc-8465-59ac413bc643 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:44.496951 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1023/2032] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:12:44 2022] PUT /identity/v3/projects/e2757dae0d3a45a29427faed00e98996/users/18728d1c3451497480556461b4ec80d5/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:12:44.504288 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0c587-f0c5-4681-a7e5-4dd44c27226a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:44.504579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0c587-f0c5-4681-a7e5-4dd44c27226a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:44.504853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0c587-f0c5-4681-a7e5-4dd44c27226a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:44.530061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-42f0c587-f0c5-4681-a7e5-4dd44c27226a None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:44.531405 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-42f0c587-f0c5-4681-a7e5-4dd44c27226a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:44.622783 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1010/2033] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:44.637860 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97f80f-abeb-4ffc-8465-59ac413bc643 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:44.639694 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1c97f80f-abeb-4ffc-8465-59ac413bc643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:44.742607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97f80f-abeb-4ffc-8465-59ac413bc643 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:44.746542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63839060-63a1-426e-8f7a-adce6b190a6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:44.747242 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63839060-63a1-426e-8f7a-adce6b190a6d service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:44.747609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63839060-63a1-426e-8f7a-adce6b190a6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:44.747806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63839060-63a1-426e-8f7a-adce6b190a6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:44.760760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63839060-63a1-426e-8f7a-adce6b190a6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:44.762427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63839060-63a1-426e-8f7a-adce6b190a6d service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:44.779936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1024/2034] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:44 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:12:49.977670 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f78b98-b3e2-42f4-a93a-d86ded1969e4 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:49.978738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f78b98-b3e2-42f4-a93a-d86ded1969e4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:49.979367 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f78b98-b3e2-42f4-a93a-d86ded1969e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:50.013480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f8f78b98-b3e2-42f4-a93a-d86ded1969e4 None None] MFA Rules not processed for user `18728d1c3451497480556461b4ec80d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:50.014731 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f8f78b98-b3e2-42f4-a93a-d86ded1969e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:50.097778 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1011/2035] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:50.114191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63839060-63a1-426e-8f7a-adce6b190a6d service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:50.115759 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-63839060-63a1-426e-8f7a-adce6b190a6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:50.264172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63839060-63a1-426e-8f7a-adce6b190a6d service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:50.273053 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe932e3-945c-4248-a485-5c04e63d0d2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:50.273790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe932e3-945c-4248-a485-5c04e63d0d2d service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:50.274083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe932e3-945c-4248-a485-5c04e63d0d2d service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:50.274367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe932e3-945c-4248-a485-5c04e63d0d2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:50.285315 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe932e3-945c-4248-a485-5c04e63d0d2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:50.287041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe932e3-945c-4248-a485-5c04e63d0d2d service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:50.298556 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1025/2036] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:12:51.813133 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3290c1-25c8-4204-9aee-8e7ad5a3dd9d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:51.813887 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3290c1-25c8-4204-9aee-8e7ad5a3dd9d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:51.813887 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3290c1-25c8-4204-9aee-8e7ad5a3dd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:51.838010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8b3290c1-25c8-4204-9aee-8e7ad5a3dd9d None None] MFA Rules not processed for user `18728d1c3451497480556461b4ec80d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:51.839413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8b3290c1-25c8-4204-9aee-8e7ad5a3dd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:51.847305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1012/2037] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:53.625519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe932e3-945c-4248-a485-5c04e63d0d2d service nova] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:53.627616 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3fe932e3-945c-4248-a485-5c04e63d0d2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:53.650969 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe932e3-945c-4248-a485-5c04e63d0d2d service nova] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:53.657675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754f262b-1186-40e5-b15f-ccdb6ac816c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:53.658400 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f262b-1186-40e5-b15f-ccdb6ac816c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:53.658690 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f262b-1186-40e5-b15f-ccdb6ac816c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:53.658971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f262b-1186-40e5-b15f-ccdb6ac816c0 admin admin] PATH_INFO: `/v3/users/bbf2980dae53403f99f4661ed9078368` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:53.660333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754f262b-1186-40e5-b15f-ccdb6ac816c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbf2980dae53403f99f4661ed9078368)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:53.662959 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754f262b-1186-40e5-b15f-ccdb6ac816c0 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:53.720621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:53.752014 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-754f262b-1186-40e5-b15f-ccdb6ac816c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbf2980dae53403f99f4661ed9078368'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:53.753693 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-754f262b-1186-40e5-b15f-ccdb6ac816c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbf2980dae53403f99f4661ed9078368'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:53.773470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-754f262b-1186-40e5-b15f-ccdb6ac816c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbf2980dae53403f99f4661ed9078368'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:53.796559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-754f262b-1186-40e5-b15f-ccdb6ac816c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbf2980dae53403f99f4661ed9078368'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:53.807416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-754f262b-1186-40e5-b15f-ccdb6ac816c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbf2980dae53403f99f4661ed9078368'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:53.820239 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1026/2038] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:12:53 2022] DELETE /identity/v3/users/bbf2980dae53403f99f4661ed9078368 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:53.821089 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab8b19-e89d-417b-90e1-3f4c5c67ebba None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:53.821670 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab8b19-e89d-417b-90e1-3f4c5c67ebba None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:53.821763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab8b19-e89d-417b-90e1-3f4c5c67ebba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:53.853917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f2ab8b19-e89d-417b-90e1-3f4c5c67ebba None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:53.855872 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f2ab8b19-e89d-417b-90e1-3f4c5c67ebba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:53.861152 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:53.933433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1013/2039] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:53.944058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754f262b-1186-40e5-b15f-ccdb6ac816c0 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:53.945546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-754f262b-1186-40e5-b15f-ccdb6ac816c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:54.018099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754f262b-1186-40e5-b15f-ccdb6ac816c0 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:54.022415 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce0fb9-578d-4465-a029-b90141466fee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:54.023294 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce0fb9-578d-4465-a029-b90141466fee service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:54.023585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce0fb9-578d-4465-a029-b90141466fee service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:54.023864 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce0fb9-578d-4465-a029-b90141466fee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:54.035092 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce0fb9-578d-4465-a029-b90141466fee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:54.036773 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce0fb9-578d-4465-a029-b90141466fee service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:54.049456 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1027/2040] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:53 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:12:54.973400 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab8b19-e89d-417b-90e1-3f4c5c67ebba None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:54.974922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f2ab8b19-e89d-417b-90e1-3f4c5c67ebba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:54.995931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab8b19-e89d-417b-90e1-3f4c5c67ebba None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:55.004408 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6816e-9abe-410e-b738-22d8ac302dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:55.005668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6816e-9abe-410e-b738-22d8ac302dd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:55.006063 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6816e-9abe-410e-b738-22d8ac302dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:55.006466 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6816e-9abe-410e-b738-22d8ac302dd3 admin admin] PATH_INFO: `/v3/projects/7e0e5bb1c55c457899677b3043941866` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:55.010848 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6816e-9abe-410e-b738-22d8ac302dd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e0e5bb1c55c457899677b3043941866)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:55.013816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6816e-9abe-410e-b738-22d8ac302dd3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:55.089421 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-80e6816e-9abe-410e-b738-22d8ac302dd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e0e5bb1c55c457899677b3043941866', 'request_id': 'req-80e6816e-9abe-410e-b738-22d8ac302dd3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:55.097323 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-80e6816e-9abe-410e-b738-22d8ac302dd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e0e5bb1c55c457899677b3043941866', 'request_id': 'req-80e6816e-9abe-410e-b738-22d8ac302dd3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:55.106432 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-80e6816e-9abe-410e-b738-22d8ac302dd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e0e5bb1c55c457899677b3043941866', 'request_id': 'req-80e6816e-9abe-410e-b738-22d8ac302dd3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:55.118971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-80e6816e-9abe-410e-b738-22d8ac302dd3 admin admin] The token cache is being invalidate because project 7e0e5bb1c55c457899677b3043941866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:12:55.119272 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-80e6816e-9abe-410e-b738-22d8ac302dd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:55.119989 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-80e6816e-9abe-410e-b738-22d8ac302dd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:55.121608 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1014/2041] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:12:54 2022] DELETE /identity/v3/projects/7e0e5bb1c55c457899677b3043941866 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:55.127374 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce0fb9-578d-4465-a029-b90141466fee service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:55.138519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce0fb9-578d-4465-a029-b90141466fee service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:55.205086 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe66b27-8435-4e5a-8f03-3a5a41ec8c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:55.205788 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe66b27-8435-4e5a-8f03-3a5a41ec8c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:55.206090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe66b27-8435-4e5a-8f03-3a5a41ec8c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:55.206369 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe66b27-8435-4e5a-8f03-3a5a41ec8c08 admin admin] PATH_INFO: `/v3/users/e2d54d5c32e24d67b97ec7a5724c9c75` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:55.207566 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe66b27-8435-4e5a-8f03-3a5a41ec8c08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2d54d5c32e24d67b97ec7a5724c9c75)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:55.209756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe66b27-8435-4e5a-8f03-3a5a41ec8c08 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:55.257663 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dfe66b27-8435-4e5a-8f03-3a5a41ec8c08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2d54d5c32e24d67b97ec7a5724c9c75'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:55.258322 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dfe66b27-8435-4e5a-8f03-3a5a41ec8c08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2d54d5c32e24d67b97ec7a5724c9c75'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:55.267303 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dfe66b27-8435-4e5a-8f03-3a5a41ec8c08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2d54d5c32e24d67b97ec7a5724c9c75'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:55.268572 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:55.282994 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dfe66b27-8435-4e5a-8f03-3a5a41ec8c08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2d54d5c32e24d67b97ec7a5724c9c75'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:55.290136 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dfe66b27-8435-4e5a-8f03-3a5a41ec8c08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2d54d5c32e24d67b97ec7a5724c9c75'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:55.297028 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1028/2042] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:12:55 2022] DELETE /identity/v3/users/e2d54d5c32e24d67b97ec7a5724c9c75 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:55.594873 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6816e-9abe-410e-b738-22d8ac302dd3 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:55.596436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-80e6816e-9abe-410e-b738-22d8ac302dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:55.611231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6816e-9abe-410e-b738-22d8ac302dd3 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:55.684416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe49ac-678e-4757-8996-f429b0d3cf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:55.685074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe49ac-678e-4757-8996-f429b0d3cf73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:55.685394 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe49ac-678e-4757-8996-f429b0d3cf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:55.685676 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe49ac-678e-4757-8996-f429b0d3cf73 admin admin] PATH_INFO: `/v3/projects/f41d5cf61d8645439d0385db0723fb89` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:55.686919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe49ac-678e-4757-8996-f429b0d3cf73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f41d5cf61d8645439d0385db0723fb89)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:55.689444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe49ac-678e-4757-8996-f429b0d3cf73 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:55.731587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:55.744975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-81fe49ac-678e-4757-8996-f429b0d3cf73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f41d5cf61d8645439d0385db0723fb89', 'request_id': 'req-81fe49ac-678e-4757-8996-f429b0d3cf73'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:55.755899 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-81fe49ac-678e-4757-8996-f429b0d3cf73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f41d5cf61d8645439d0385db0723fb89', 'request_id': 'req-81fe49ac-678e-4757-8996-f429b0d3cf73'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:55.764694 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-81fe49ac-678e-4757-8996-f429b0d3cf73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f41d5cf61d8645439d0385db0723fb89', 'request_id': 'req-81fe49ac-678e-4757-8996-f429b0d3cf73'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:55.777883 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-81fe49ac-678e-4757-8996-f429b0d3cf73 admin admin] The token cache is being invalidate because project f41d5cf61d8645439d0385db0723fb89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:12:55.778180 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-81fe49ac-678e-4757-8996-f429b0d3cf73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:55.778847 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-81fe49ac-678e-4757-8996-f429b0d3cf73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:55.780428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1015/2043] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:12:55 2022] DELETE /identity/v3/projects/f41d5cf61d8645439d0385db0723fb89 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:55.786054 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21791f18-a8c5-48bc-8303-5b2fdcc34543 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:55.786568 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21791f18-a8c5-48bc-8303-5b2fdcc34543 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:55.786948 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21791f18-a8c5-48bc-8303-5b2fdcc34543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:55.807744 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-21791f18-a8c5-48bc-8303-5b2fdcc34543 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:55.809043 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-21791f18-a8c5-48bc-8303-5b2fdcc34543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:55.875842 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1029/2044] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:55.883925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73567db1-a971-4bc3-962a-c337bd5d8e15 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:55.884212 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73567db1-a971-4bc3-962a-c337bd5d8e15 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:55.884483 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73567db1-a971-4bc3-962a-c337bd5d8e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:55.905004 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-73567db1-a971-4bc3-962a-c337bd5d8e15 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:55.907388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-73567db1-a971-4bc3-962a-c337bd5d8e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:55.915381 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1016/2045] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:12:55.919858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21791f18-a8c5-48bc-8303-5b2fdcc34543 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:55.934420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21791f18-a8c5-48bc-8303-5b2fdcc34543 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:55.939598 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70250d2-f69c-4871-a3ea-0039f7536dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:55.940230 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70250d2-f69c-4871-a3ea-0039f7536dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:55.940553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70250d2-f69c-4871-a3ea-0039f7536dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:55.940828 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70250d2-f69c-4871-a3ea-0039f7536dd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:55.941573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70250d2-f69c-4871-a3ea-0039f7536dd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:12:55.941880 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70250d2-f69c-4871-a3ea-0039f7536dd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:55.943558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70250d2-f69c-4871-a3ea-0039f7536dd3 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:55.957275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1030/2046] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:12:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:12:55.963288 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73567db1-a971-4bc3-962a-c337bd5d8e15 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:55.975062 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73567db1-a971-4bc3-962a-c337bd5d8e15 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:55.980044 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3e214-8d08-40c6-8e6d-9d965a095e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:55.980682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3e214-8d08-40c6-8e6d-9d965a095e97 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:55.980966 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3e214-8d08-40c6-8e6d-9d965a095e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:55.981278 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3e214-8d08-40c6-8e6d-9d965a095e97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:55.981961 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3e214-8d08-40c6-8e6d-9d965a095e97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:55.984531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3e214-8d08-40c6-8e6d-9d965a095e97 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:56.019960 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1017/2047] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:12:55 2022] POST /identity/v3/projects => generated 380 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:12:56.029955 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70250d2-f69c-4871-a3ea-0039f7536dd3 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:56.041913 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70250d2-f69c-4871-a3ea-0039f7536dd3 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:56.126177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86a32e-63d1-411b-9b87-4452aae68ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:56.126819 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86a32e-63d1-411b-9b87-4452aae68ab9 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:56.127184 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86a32e-63d1-411b-9b87-4452aae68ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:56.127462 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86a32e-63d1-411b-9b87-4452aae68ab9 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:56.128133 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86a32e-63d1-411b-9b87-4452aae68ab9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:56.130406 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86a32e-63d1-411b-9b87-4452aae68ab9 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:56.162112 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1031/2048] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:12:56 2022] POST /identity/v3/users => generated 291 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:12:56.166984 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3e214-8d08-40c6-8e6d-9d965a095e97 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:56.181684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3e214-8d08-40c6-8e6d-9d965a095e97 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:56.187077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf1ce9-39cc-4802-afeb-3f778538264f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:56.187728 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf1ce9-39cc-4802-afeb-3f778538264f admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:56.188116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf1ce9-39cc-4802-afeb-3f778538264f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:56.188582 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf1ce9-39cc-4802-afeb-3f778538264f admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:56.189333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf1ce9-39cc-4802-afeb-3f778538264f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:56.190963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf1ce9-39cc-4802-afeb-3f778538264f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:56.206385 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1018/2049] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:56 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:56.211444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86a32e-63d1-411b-9b87-4452aae68ab9 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:56.223615 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86a32e-63d1-411b-9b87-4452aae68ab9 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:56.228690 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd299ebe-accd-4119-a6eb-79141603cdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:56.229675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd299ebe-accd-4119-a6eb-79141603cdfa admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:56.229953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd299ebe-accd-4119-a6eb-79141603cdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:56.230482 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd299ebe-accd-4119-a6eb-79141603cdfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:56.231325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd299ebe-accd-4119-a6eb-79141603cdfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:56.233134 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd299ebe-accd-4119-a6eb-79141603cdfa admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:56.246994 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1032/2050] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:56 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:56.251703 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf1ce9-39cc-4802-afeb-3f778538264f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:56.264537 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf1ce9-39cc-4802-afeb-3f778538264f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:56.270783 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03642c1c-2241-4191-a05f-c3e2831aeae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:56.271506 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03642c1c-2241-4191-a05f-c3e2831aeae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:56.271796 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03642c1c-2241-4191-a05f-c3e2831aeae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:56.272100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03642c1c-2241-4191-a05f-c3e2831aeae8 admin admin] PATH_INFO: `/v3/projects/616c81d297784d64bd129bf1ad298ad4/users/d63cb3e6078d4ed5a68f0f6d2bc7c68a/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:56.291620 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03642c1c-2241-4191-a05f-c3e2831aeae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=616c81d297784d64bd129bf1ad298ad4, user_id=d63cb3e6078d4ed5a68f0f6d2bc7c68a, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:56.294654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03642c1c-2241-4191-a05f-c3e2831aeae8 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:56.321498 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1019/2051] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:12:56 2022] PUT /identity/v3/projects/616c81d297784d64bd129bf1ad298ad4/users/d63cb3e6078d4ed5a68f0f6d2bc7c68a/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:12:56.330920 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7353e32-da54-4de6-ab88-54c09d2f2077 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:56.331567 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7353e32-da54-4de6-ab88-54c09d2f2077 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:56.331948 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7353e32-da54-4de6-ab88-54c09d2f2077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:56.373311 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-d7353e32-da54-4de6-ab88-54c09d2f2077 None None] MFA Rules not processed for user `d63cb3e6078d4ed5a68f0f6d2bc7c68a`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:56.374666 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d7353e32-da54-4de6-ab88-54c09d2f2077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:56.410383 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:56.475163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1033/2052] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:56.479395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03642c1c-2241-4191-a05f-c3e2831aeae8 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:56.558442 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03642c1c-2241-4191-a05f-c3e2831aeae8 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:56.563367 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564578b9-739e-4f7b-987e-7c9463a4a309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:56.564003 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564578b9-739e-4f7b-987e-7c9463a4a309 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:56.564322 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564578b9-739e-4f7b-987e-7c9463a4a309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:56.564727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564578b9-739e-4f7b-987e-7c9463a4a309 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:56.565495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564578b9-739e-4f7b-987e-7c9463a4a309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:56.567418 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564578b9-739e-4f7b-987e-7c9463a4a309 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:56.600297 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1020/2053] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:12:56 2022] POST /identity/v3/projects => generated 382 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:12:56.604438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7353e32-da54-4de6-ab88-54c09d2f2077 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:56.618025 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7353e32-da54-4de6-ab88-54c09d2f2077 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:56.689531 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448112b-db09-4f02-bdcc-ca8b0235b547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:56.690108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448112b-db09-4f02-bdcc-ca8b0235b547 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:56.690392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448112b-db09-4f02-bdcc-ca8b0235b547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:56.690672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448112b-db09-4f02-bdcc-ca8b0235b547 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:56.691352 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448112b-db09-4f02-bdcc-ca8b0235b547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:56.693349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448112b-db09-4f02-bdcc-ca8b0235b547 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:56.721223 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1034/2054] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:12:56 2022] POST /identity/v3/users => generated 298 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:12:56.725077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564578b9-739e-4f7b-987e-7c9463a4a309 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:56.739320 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564578b9-739e-4f7b-987e-7c9463a4a309 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:56.746688 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353fd4a-1cc3-453b-afe4-0d3b865ff1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:56.747424 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353fd4a-1cc3-453b-afe4-0d3b865ff1a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:56.747772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353fd4a-1cc3-453b-afe4-0d3b865ff1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:56.748113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353fd4a-1cc3-453b-afe4-0d3b865ff1a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:56.748904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353fd4a-1cc3-453b-afe4-0d3b865ff1a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:56.750801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353fd4a-1cc3-453b-afe4-0d3b865ff1a1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:56.765135 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1021/2055] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:56 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:56.769640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448112b-db09-4f02-bdcc-ca8b0235b547 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:56.782559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448112b-db09-4f02-bdcc-ca8b0235b547 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:56.787538 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe33f1a-1de1-42ed-a572-1988bffb48d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:56.788215 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe33f1a-1de1-42ed-a572-1988bffb48d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:56.788498 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe33f1a-1de1-42ed-a572-1988bffb48d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:56.788776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe33f1a-1de1-42ed-a572-1988bffb48d5 admin admin] PATH_INFO: `/v3/projects/6cc1c4a8343948efadb43bb317c9ec69/users/a55b53c57afa4e97ad4a3160ee610b92/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:56.808404 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe33f1a-1de1-42ed-a572-1988bffb48d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cc1c4a8343948efadb43bb317c9ec69, user_id=a55b53c57afa4e97ad4a3160ee610b92, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:56.811237 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe33f1a-1de1-42ed-a572-1988bffb48d5 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:56.829895 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1035/2056] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:12:56 2022] PUT /identity/v3/projects/6cc1c4a8343948efadb43bb317c9ec69/users/a55b53c57afa4e97ad4a3160ee610b92/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:56.838244 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353fd4a-1cc3-453b-afe4-0d3b865ff1a1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:56.853401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353fd4a-1cc3-453b-afe4-0d3b865ff1a1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:56.903413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:56.951723 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7a7e5-9462-4e7d-92b1-58f3517fba53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:56.954365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7a7e5-9462-4e7d-92b1-58f3517fba53 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:56.954731 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7a7e5-9462-4e7d-92b1-58f3517fba53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:56.955107 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7a7e5-9462-4e7d-92b1-58f3517fba53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:56.955992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7a7e5-9462-4e7d-92b1-58f3517fba53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:56.957933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7a7e5-9462-4e7d-92b1-58f3517fba53 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:56.973710 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1022/2057] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:56 2022] GET /identity/v3/roles => generated 1427 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:56.974990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe33f1a-1de1-42ed-a572-1988bffb48d5 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:56.989545 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe33f1a-1de1-42ed-a572-1988bffb48d5 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:56.995118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea3e82-efb6-4c86-8564-38c64a94e518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:56.996208 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea3e82-efb6-4c86-8564-38c64a94e518 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:56.996609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea3e82-efb6-4c86-8564-38c64a94e518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:56.996958 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea3e82-efb6-4c86-8564-38c64a94e518 admin admin] PATH_INFO: `/v3/projects/6cc1c4a8343948efadb43bb317c9ec69/users/a55b53c57afa4e97ad4a3160ee610b92/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:56.999299 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea3e82-efb6-4c86-8564-38c64a94e518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cc1c4a8343948efadb43bb317c9ec69, user_id=a55b53c57afa4e97ad4a3160ee610b92, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:57.003726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea3e82-efb6-4c86-8564-38c64a94e518 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:57.018513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1036/2058] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:12:56 2022] PUT /identity/v3/projects/6cc1c4a8343948efadb43bb317c9ec69/users/a55b53c57afa4e97ad4a3160ee610b92/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:57.028698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0554d1-00fd-4c7f-be05-ba764715ccb2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:57.029275 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0554d1-00fd-4c7f-be05-ba764715ccb2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:57.029275 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0554d1-00fd-4c7f-be05-ba764715ccb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:57.060806 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-bc0554d1-00fd-4c7f-be05-ba764715ccb2 None None] MFA Rules not processed for user `a55b53c57afa4e97ad4a3160ee610b92`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:57.062281 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bc0554d1-00fd-4c7f-be05-ba764715ccb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:57.143803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1023/2059] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:57 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:57.160612 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea3e82-efb6-4c86-8564-38c64a94e518 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:57.162624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-eaea3e82-efb6-4c86-8564-38c64a94e518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:57.234175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea3e82-efb6-4c86-8564-38c64a94e518 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:57.239935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbd33f-909d-437b-bcfc-38a591a682e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:57.240743 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbd33f-909d-437b-bcfc-38a591a682e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:57.241147 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbd33f-909d-437b-bcfc-38a591a682e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:57.241981 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbd33f-909d-437b-bcfc-38a591a682e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:57.258358 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbd33f-909d-437b-bcfc-38a591a682e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:57.260446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbd33f-909d-437b-bcfc-38a591a682e5 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:57.278406 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1037/2060] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:57 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:12:57.818459 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0554d1-00fd-4c7f-be05-ba764715ccb2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:57.890541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0554d1-00fd-4c7f-be05-ba764715ccb2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:57.895198 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f918f-abdf-4f36-bb3d-5e6e55145fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:57.895862 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f918f-abdf-4f36-bb3d-5e6e55145fed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:57.896148 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f918f-abdf-4f36-bb3d-5e6e55145fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:57.896427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f918f-abdf-4f36-bb3d-5e6e55145fed admin admin] PATH_INFO: `/v3/users/d63cb3e6078d4ed5a68f0f6d2bc7c68a` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:57.897624 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f918f-abdf-4f36-bb3d-5e6e55145fed admin admin] RBAC: Authorizing `identity:delete_user(user_id=d63cb3e6078d4ed5a68f0f6d2bc7c68a)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:57.899559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f918f-abdf-4f36-bb3d-5e6e55145fed admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:57.951346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5c2f918f-abdf-4f36-bb3d-5e6e55145fed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd63cb3e6078d4ed5a68f0f6d2bc7c68a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:57.958860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5c2f918f-abdf-4f36-bb3d-5e6e55145fed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd63cb3e6078d4ed5a68f0f6d2bc7c68a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:57.959596 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5c2f918f-abdf-4f36-bb3d-5e6e55145fed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd63cb3e6078d4ed5a68f0f6d2bc7c68a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:57.961375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:57.969175 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5c2f918f-abdf-4f36-bb3d-5e6e55145fed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd63cb3e6078d4ed5a68f0f6d2bc7c68a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:57.970010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5c2f918f-abdf-4f36-bb3d-5e6e55145fed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd63cb3e6078d4ed5a68f0f6d2bc7c68a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:57.990086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1024/2061] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:12:57 2022] DELETE /identity/v3/users/d63cb3e6078d4ed5a68f0f6d2bc7c68a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:12:57.994925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923fee8-864e-4fc9-94ca-2a8ce3a6976c None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:57.995218 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923fee8-864e-4fc9-94ca-2a8ce3a6976c None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:57.995499 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923fee8-864e-4fc9-94ca-2a8ce3a6976c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:58.020970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-e923fee8-864e-4fc9-94ca-2a8ce3a6976c None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:58.022216 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e923fee8-864e-4fc9-94ca-2a8ce3a6976c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:58.040031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:58.086578 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1038/2062] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:58.096698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f918f-abdf-4f36-bb3d-5e6e55145fed admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:58.098202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5c2f918f-abdf-4f36-bb3d-5e6e55145fed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:58.157912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f918f-abdf-4f36-bb3d-5e6e55145fed admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:58.161642 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f0689-8a56-47fc-8583-ca5ecdf27942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:58.162268 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f0689-8a56-47fc-8583-ca5ecdf27942 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:58.162553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f0689-8a56-47fc-8583-ca5ecdf27942 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:58.162831 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f0689-8a56-47fc-8583-ca5ecdf27942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:58.174793 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f0689-8a56-47fc-8583-ca5ecdf27942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:58.176504 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f0689-8a56-47fc-8583-ca5ecdf27942 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:58.190225 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1025/2063] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:12:58 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:12:58.521260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923fee8-864e-4fc9-94ca-2a8ce3a6976c None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:58.523165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e923fee8-864e-4fc9-94ca-2a8ce3a6976c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:58.535526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923fee8-864e-4fc9-94ca-2a8ce3a6976c None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:58.540242 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353d069-f245-4188-b6ca-baaf8182f655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:58.540930 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353d069-f245-4188-b6ca-baaf8182f655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:58.541252 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353d069-f245-4188-b6ca-baaf8182f655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:58.541536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353d069-f245-4188-b6ca-baaf8182f655 admin admin] PATH_INFO: `/v3/projects/616c81d297784d64bd129bf1ad298ad4` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:58.543347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353d069-f245-4188-b6ca-baaf8182f655 admin admin] RBAC: Authorizing `identity:delete_project(project_id=616c81d297784d64bd129bf1ad298ad4)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:58.545346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353d069-f245-4188-b6ca-baaf8182f655 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:58.633360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3353d069-f245-4188-b6ca-baaf8182f655 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '616c81d297784d64bd129bf1ad298ad4', 'request_id': 'req-3353d069-f245-4188-b6ca-baaf8182f655'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:58.644953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3353d069-f245-4188-b6ca-baaf8182f655 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '616c81d297784d64bd129bf1ad298ad4', 'request_id': 'req-3353d069-f245-4188-b6ca-baaf8182f655'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:58.655312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3353d069-f245-4188-b6ca-baaf8182f655 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '616c81d297784d64bd129bf1ad298ad4', 'request_id': 'req-3353d069-f245-4188-b6ca-baaf8182f655'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:58.668699 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3353d069-f245-4188-b6ca-baaf8182f655 admin admin] The token cache is being invalidate because project 616c81d297784d64bd129bf1ad298ad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:12:58.669027 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3353d069-f245-4188-b6ca-baaf8182f655 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:58.669705 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3353d069-f245-4188-b6ca-baaf8182f655 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:58.671834 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1039/2064] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:12:58 2022] DELETE /identity/v3/projects/616c81d297784d64bd129bf1ad298ad4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:58.676309 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f0689-8a56-47fc-8583-ca5ecdf27942 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:58.691652 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f0689-8a56-47fc-8583-ca5ecdf27942 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:58.781859 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb302376-21a0-4925-922b-49ffa9fe0d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:58.782748 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb302376-21a0-4925-922b-49ffa9fe0d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:58.783053 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb302376-21a0-4925-922b-49ffa9fe0d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:58.783405 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb302376-21a0-4925-922b-49ffa9fe0d39 admin admin] PATH_INFO: `/v3/users/a55b53c57afa4e97ad4a3160ee610b92` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:58.784816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb302376-21a0-4925-922b-49ffa9fe0d39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a55b53c57afa4e97ad4a3160ee610b92)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:58.786986 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb302376-21a0-4925-922b-49ffa9fe0d39 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:58.852231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-eb302376-21a0-4925-922b-49ffa9fe0d39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a55b53c57afa4e97ad4a3160ee610b92'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:58.859508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-eb302376-21a0-4925-922b-49ffa9fe0d39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a55b53c57afa4e97ad4a3160ee610b92'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:58.860292 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-eb302376-21a0-4925-922b-49ffa9fe0d39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a55b53c57afa4e97ad4a3160ee610b92'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:58.867997 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-eb302376-21a0-4925-922b-49ffa9fe0d39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a55b53c57afa4e97ad4a3160ee610b92'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:58.868623 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-eb302376-21a0-4925-922b-49ffa9fe0d39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a55b53c57afa4e97ad4a3160ee610b92'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:58.880963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:58.889270 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1026/2065] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:12:58 2022] DELETE /identity/v3/users/a55b53c57afa4e97ad4a3160ee610b92 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:12:59.192976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353d069-f245-4188-b6ca-baaf8182f655 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:59.194612 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3353d069-f245-4188-b6ca-baaf8182f655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:59.208558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353d069-f245-4188-b6ca-baaf8182f655 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:59.275102 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76414d26-e6b7-4c54-8bd4-f5bf50611bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:59.275810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76414d26-e6b7-4c54-8bd4-f5bf50611bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:59.276123 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76414d26-e6b7-4c54-8bd4-f5bf50611bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:59.276397 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76414d26-e6b7-4c54-8bd4-f5bf50611bd6 admin admin] PATH_INFO: `/v3/projects/6cc1c4a8343948efadb43bb317c9ec69` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:59.277707 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76414d26-e6b7-4c54-8bd4-f5bf50611bd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cc1c4a8343948efadb43bb317c9ec69)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:59.279684 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76414d26-e6b7-4c54-8bd4-f5bf50611bd6 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:59.327621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:59.329671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-76414d26-e6b7-4c54-8bd4-f5bf50611bd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cc1c4a8343948efadb43bb317c9ec69', 'request_id': 'req-76414d26-e6b7-4c54-8bd4-f5bf50611bd6'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:59.337308 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-76414d26-e6b7-4c54-8bd4-f5bf50611bd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cc1c4a8343948efadb43bb317c9ec69', 'request_id': 'req-76414d26-e6b7-4c54-8bd4-f5bf50611bd6'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:59.351161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-76414d26-e6b7-4c54-8bd4-f5bf50611bd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cc1c4a8343948efadb43bb317c9ec69', 'request_id': 'req-76414d26-e6b7-4c54-8bd4-f5bf50611bd6'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:59.365276 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-76414d26-e6b7-4c54-8bd4-f5bf50611bd6 admin admin] The token cache is being invalidate because project 6cc1c4a8343948efadb43bb317c9ec69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:12:59.365582 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-76414d26-e6b7-4c54-8bd4-f5bf50611bd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:59.366264 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-76414d26-e6b7-4c54-8bd4-f5bf50611bd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:59.367793 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1040/2066] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:12:59 2022] DELETE /identity/v3/projects/6cc1c4a8343948efadb43bb317c9ec69 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:12:59.372738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d63c66-96e0-4668-a7c9-1b588dd8961b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:59.373350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d63c66-96e0-4668-a7c9-1b588dd8961b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:59.373526 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d63c66-96e0-4668-a7c9-1b588dd8961b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:59.395453 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-63d63c66-96e0-4668-a7c9-1b588dd8961b None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:59.396936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-63d63c66-96e0-4668-a7c9-1b588dd8961b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:59.464701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1027/2067] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:59 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:59.473327 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8681d73-18b3-4de3-9796-a1e0d1501cd0 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:59.473612 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8681d73-18b3-4de3-9796-a1e0d1501cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:59.473906 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8681d73-18b3-4de3-9796-a1e0d1501cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:59.495782 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-c8681d73-18b3-4de3-9796-a1e0d1501cd0 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:59.497016 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c8681d73-18b3-4de3-9796-a1e0d1501cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:59.506298 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1041/2068] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:59 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:59.510260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d63c66-96e0-4668-a7c9-1b588dd8961b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:59.520976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d63c66-96e0-4668-a7c9-1b588dd8961b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:59.526287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0760c46-d656-42b9-8378-02780e9ab744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:59.526933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0760c46-d656-42b9-8378-02780e9ab744 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:59.527217 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0760c46-d656-42b9-8378-02780e9ab744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:59.527500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0760c46-d656-42b9-8378-02780e9ab744 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:59.528206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0760c46-d656-42b9-8378-02780e9ab744 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:12:59.528514 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0760c46-d656-42b9-8378-02780e9ab744 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:59.530202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0760c46-d656-42b9-8378-02780e9ab744 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:59.544480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1028/2069] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:12:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:12:59.549737 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8681d73-18b3-4de3-9796-a1e0d1501cd0 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:59.560072 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8681d73-18b3-4de3-9796-a1e0d1501cd0 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:59.565858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a47de1-9752-497e-ad6f-d20d344fedbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:59.566437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a47de1-9752-497e-ad6f-d20d344fedbc admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:59.566717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a47de1-9752-497e-ad6f-d20d344fedbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:59.566990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a47de1-9752-497e-ad6f-d20d344fedbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:59.567673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a47de1-9752-497e-ad6f-d20d344fedbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:59.569632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a47de1-9752-497e-ad6f-d20d344fedbc admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:59.601787 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1042/2070] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:12:59 2022] POST /identity/v3/projects => generated 384 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:12:59.605980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0760c46-d656-42b9-8378-02780e9ab744 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:59.618720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0760c46-d656-42b9-8378-02780e9ab744 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:59.684896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ee3ec-8afc-4714-99eb-f4b7c173bb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:59.685558 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ee3ec-8afc-4714-99eb-f4b7c173bb99 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:59.685865 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ee3ec-8afc-4714-99eb-f4b7c173bb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:59.686142 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ee3ec-8afc-4714-99eb-f4b7c173bb99 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:59.687435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ee3ec-8afc-4714-99eb-f4b7c173bb99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:59.689624 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ee3ec-8afc-4714-99eb-f4b7c173bb99 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:59.715163 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1029/2071] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:12:59 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:12:59.719454 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a47de1-9752-497e-ad6f-d20d344fedbc admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:59.730224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a47de1-9752-497e-ad6f-d20d344fedbc admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:59.735167 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153e21c-be2e-4bc0-a136-e43eb0ab8769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:59.735793 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153e21c-be2e-4bc0-a136-e43eb0ab8769 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:59.736070 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153e21c-be2e-4bc0-a136-e43eb0ab8769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:59.736340 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153e21c-be2e-4bc0-a136-e43eb0ab8769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:59.737168 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153e21c-be2e-4bc0-a136-e43eb0ab8769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:59.738783 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153e21c-be2e-4bc0-a136-e43eb0ab8769 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:59.750781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1043/2072] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:59 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:59.755026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ee3ec-8afc-4714-99eb-f4b7c173bb99 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:59.766451 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ee3ec-8afc-4714-99eb-f4b7c173bb99 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:59.771913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d54e3e-f5df-473e-87cd-2c2725b1e4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:59.772546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d54e3e-f5df-473e-87cd-2c2725b1e4ee admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:59.772832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d54e3e-f5df-473e-87cd-2c2725b1e4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:59.773145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d54e3e-f5df-473e-87cd-2c2725b1e4ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:59.773869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d54e3e-f5df-473e-87cd-2c2725b1e4ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:59.775470 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d54e3e-f5df-473e-87cd-2c2725b1e4ee admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:59.787131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1030/2073] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:12:59 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:12:59.793356 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153e21c-be2e-4bc0-a136-e43eb0ab8769 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:59.804254 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153e21c-be2e-4bc0-a136-e43eb0ab8769 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:59.809747 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100f0cc-b7cc-4709-940d-48b51c137dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:59.810459 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100f0cc-b7cc-4709-940d-48b51c137dde admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:59.810720 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100f0cc-b7cc-4709-940d-48b51c137dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:59.811000 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100f0cc-b7cc-4709-940d-48b51c137dde admin admin] PATH_INFO: `/v3/projects/62aa4e761f5f4a2f8adf0d75d587ca62/users/d960d57ac8f743d798ca32a21b87cf16/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:59.830808 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2100f0cc-b7cc-4709-940d-48b51c137dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=62aa4e761f5f4a2f8adf0d75d587ca62, user_id=d960d57ac8f743d798ca32a21b87cf16, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:59.835445 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2100f0cc-b7cc-4709-940d-48b51c137dde admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:59.850091 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1044/2074] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:12:59 2022] PUT /identity/v3/projects/62aa4e761f5f4a2f8adf0d75d587ca62/users/d960d57ac8f743d798ca32a21b87cf16/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:59.862141 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517688c-cad8-4437-bdf8-ced50c099501 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:59.862330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517688c-cad8-4437-bdf8-ced50c099501 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:59.862621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517688c-cad8-4437-bdf8-ced50c099501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:59.913144 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f517688c-cad8-4437-bdf8-ced50c099501 None None] MFA Rules not processed for user `d960d57ac8f743d798ca32a21b87cf16`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:59.914433 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f517688c-cad8-4437-bdf8-ced50c099501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:59.989306 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:00.010217 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1031/2075] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:12:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:00.012786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100f0cc-b7cc-4709-940d-48b51c137dde admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:00.079772 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100f0cc-b7cc-4709-940d-48b51c137dde admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:00.084709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630406b-1315-46d8-8ddf-f67c9d61310c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:00.085369 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630406b-1315-46d8-8ddf-f67c9d61310c admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:00.085652 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630406b-1315-46d8-8ddf-f67c9d61310c admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:00.085928 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630406b-1315-46d8-8ddf-f67c9d61310c admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:00.086598 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630406b-1315-46d8-8ddf-f67c9d61310c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:00.088495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630406b-1315-46d8-8ddf-f67c9d61310c admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:00.122344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1045/2076] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:13:00 2022] POST /identity/v3/projects => generated 384 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:00.123575 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517688c-cad8-4437-bdf8-ced50c099501 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:00.136410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517688c-cad8-4437-bdf8-ced50c099501 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:00.217045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74005e2-c9a0-4372-abf6-1ae2e49599b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:00.217860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74005e2-c9a0-4372-abf6-1ae2e49599b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:00.218988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74005e2-c9a0-4372-abf6-1ae2e49599b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:00.219231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74005e2-c9a0-4372-abf6-1ae2e49599b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:00.220038 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74005e2-c9a0-4372-abf6-1ae2e49599b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:00.222272 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74005e2-c9a0-4372-abf6-1ae2e49599b2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:00.247637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1032/2077] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:13:00 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:13:00.250187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630406b-1315-46d8-8ddf-f67c9d61310c admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:00.261477 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630406b-1315-46d8-8ddf-f67c9d61310c admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:00.267068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a7191-3ef4-4a56-9349-a6eb21155c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:00.268035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a7191-3ef4-4a56-9349-a6eb21155c28 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:00.268035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a7191-3ef4-4a56-9349-a6eb21155c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:00.268229 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a7191-3ef4-4a56-9349-a6eb21155c28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:00.268928 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7a7191-3ef4-4a56-9349-a6eb21155c28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:00.270898 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7a7191-3ef4-4a56-9349-a6eb21155c28 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:00.287485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1046/2078] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:13:00 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:00.288894 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74005e2-c9a0-4372-abf6-1ae2e49599b2 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:00.300521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74005e2-c9a0-4372-abf6-1ae2e49599b2 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:00.307067 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef6779-7299-4a9f-adbf-d5efd83785d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:00.307903 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef6779-7299-4a9f-adbf-d5efd83785d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:00.308324 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef6779-7299-4a9f-adbf-d5efd83785d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:00.308692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef6779-7299-4a9f-adbf-d5efd83785d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:00.309661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef6779-7299-4a9f-adbf-d5efd83785d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:00.312234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef6779-7299-4a9f-adbf-d5efd83785d1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:00.329263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1033/2079] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:13:00 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:00.330510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a7191-3ef4-4a56-9349-a6eb21155c28 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:00.341381 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a7191-3ef4-4a56-9349-a6eb21155c28 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:00.346678 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e458d-7380-4f2b-a059-b6f55037d364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:00.347361 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e458d-7380-4f2b-a059-b6f55037d364 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:00.347648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e458d-7380-4f2b-a059-b6f55037d364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:00.347929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e458d-7380-4f2b-a059-b6f55037d364 admin admin] PATH_INFO: `/v3/projects/c2c133b1e6874f47828cf55bc2aedd9c/users/6d291903696749828f70df314ecbe6e5/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:00.365919 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e458d-7380-4f2b-a059-b6f55037d364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2c133b1e6874f47828cf55bc2aedd9c, user_id=6d291903696749828f70df314ecbe6e5, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:00.368390 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e458d-7380-4f2b-a059-b6f55037d364 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:00.386026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1047/2080] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:13:00 2022] PUT /identity/v3/projects/c2c133b1e6874f47828cf55bc2aedd9c/users/6d291903696749828f70df314ecbe6e5/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:00.396598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282ce43-5b13-4b59-bc6d-a818f722bb8c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:00.397153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282ce43-5b13-4b59-bc6d-a818f722bb8c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:00.397329 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282ce43-5b13-4b59-bc6d-a818f722bb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:00.430853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3282ce43-5b13-4b59-bc6d-a818f722bb8c None None] MFA Rules not processed for user `6d291903696749828f70df314ecbe6e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:00.432084 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3282ce43-5b13-4b59-bc6d-a818f722bb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:00.504971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1034/2081] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:00.512162 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e458d-7380-4f2b-a059-b6f55037d364 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:00.539894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:00.586836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e458d-7380-4f2b-a059-b6f55037d364 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:00.591828 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869facb-c173-4748-b809-6993e30cac83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:00.592459 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869facb-c173-4748-b809-6993e30cac83 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:00.592737 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869facb-c173-4748-b809-6993e30cac83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:00.593069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869facb-c173-4748-b809-6993e30cac83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:00.596850 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869facb-c173-4748-b809-6993e30cac83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:00.598607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869facb-c173-4748-b809-6993e30cac83 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:00.637351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1048/2082] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:13:00 2022] POST /identity/v3/projects => generated 384 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:13:00.639193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282ce43-5b13-4b59-bc6d-a818f722bb8c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:00.658973 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282ce43-5b13-4b59-bc6d-a818f722bb8c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:00.742798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae725d-b1cd-4e81-80a0-fc6c67529686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:00.743037 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae725d-b1cd-4e81-80a0-fc6c67529686 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:00.743962 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae725d-b1cd-4e81-80a0-fc6c67529686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:00.743962 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae725d-b1cd-4e81-80a0-fc6c67529686 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:00.744589 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae725d-b1cd-4e81-80a0-fc6c67529686 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:00.747539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae725d-b1cd-4e81-80a0-fc6c67529686 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:00.759905 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:00.783978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1035/2083] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:13:00 2022] POST /identity/v3/users => generated 299 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:00.785524 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869facb-c173-4748-b809-6993e30cac83 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:00.802279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869facb-c173-4748-b809-6993e30cac83 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:00.809197 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b1e9d-8e52-486f-b09d-bffa5fd1c2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:00.809837 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b1e9d-8e52-486f-b09d-bffa5fd1c2ae admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:00.810124 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b1e9d-8e52-486f-b09d-bffa5fd1c2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:00.810480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b1e9d-8e52-486f-b09d-bffa5fd1c2ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:00.811191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b1e9d-8e52-486f-b09d-bffa5fd1c2ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:00.812984 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b1e9d-8e52-486f-b09d-bffa5fd1c2ae admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:00.826402 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1049/2084] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:13:00 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:00.832009 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae725d-b1cd-4e81-80a0-fc6c67529686 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:00.844594 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae725d-b1cd-4e81-80a0-fc6c67529686 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:00.850013 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c36b38-93ca-4ba1-be98-444cd1393021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:00.850722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c36b38-93ca-4ba1-be98-444cd1393021 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:00.851032 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c36b38-93ca-4ba1-be98-444cd1393021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:00.851312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c36b38-93ca-4ba1-be98-444cd1393021 admin admin] PATH_INFO: `/v3/projects/3a2df9e33aa043f4816b2d15fa76ad86/users/63522869a31746ffa2e6aed07af37a2c/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:00.870316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c36b38-93ca-4ba1-be98-444cd1393021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a2df9e33aa043f4816b2d15fa76ad86, user_id=63522869a31746ffa2e6aed07af37a2c, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:00.873532 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c36b38-93ca-4ba1-be98-444cd1393021 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:00.899962 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1036/2085] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:13:00 2022] PUT /identity/v3/projects/3a2df9e33aa043f4816b2d15fa76ad86/users/63522869a31746ffa2e6aed07af37a2c/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:13:00.908204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b1e9d-8e52-486f-b09d-bffa5fd1c2ae admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:00.937884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b1e9d-8e52-486f-b09d-bffa5fd1c2ae admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:01.045085 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208fa97-80d3-4cd2-aa69-cc59dd881ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:01.068086 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208fa97-80d3-4cd2-aa69-cc59dd881ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:01.068647 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208fa97-80d3-4cd2-aa69-cc59dd881ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:01.068969 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208fa97-80d3-4cd2-aa69-cc59dd881ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:01.072559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208fa97-80d3-4cd2-aa69-cc59dd881ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:01.075486 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208fa97-80d3-4cd2-aa69-cc59dd881ec4 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:01.092186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1050/2086] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:13:00 2022] GET /identity/v3/roles => generated 1427 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:01.099065 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c36b38-93ca-4ba1-be98-444cd1393021 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:01.114685 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c36b38-93ca-4ba1-be98-444cd1393021 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:01.121470 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b8bcb-379c-4826-ae6c-8047aec00efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:01.122191 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b8bcb-379c-4826-ae6c-8047aec00efb admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:01.122580 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b8bcb-379c-4826-ae6c-8047aec00efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:01.122875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b8bcb-379c-4826-ae6c-8047aec00efb admin admin] PATH_INFO: `/v3/projects/3a2df9e33aa043f4816b2d15fa76ad86/users/63522869a31746ffa2e6aed07af37a2c/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:01.128432 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b8bcb-379c-4826-ae6c-8047aec00efb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a2df9e33aa043f4816b2d15fa76ad86, user_id=63522869a31746ffa2e6aed07af37a2c, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:01.131906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b8bcb-379c-4826-ae6c-8047aec00efb admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:01.157713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1037/2087] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:13:01 2022] PUT /identity/v3/projects/3a2df9e33aa043f4816b2d15fa76ad86/users/63522869a31746ffa2e6aed07af37a2c/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:01.169506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88720a6-67f3-4aae-a6a6-8921d5799982 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:01.169850 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88720a6-67f3-4aae-a6a6-8921d5799982 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:01.170171 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88720a6-67f3-4aae-a6a6-8921d5799982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:01.254338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-a88720a6-67f3-4aae-a6a6-8921d5799982 None None] MFA Rules not processed for user `63522869a31746ffa2e6aed07af37a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:01.255943 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-a88720a6-67f3-4aae-a6a6-8921d5799982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:01.369601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1051/2088] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:01 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:01.394718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b8bcb-379c-4826-ae6c-8047aec00efb admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:01.397888 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f42b8bcb-379c-4826-ae6c-8047aec00efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:01.479666 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b8bcb-379c-4826-ae6c-8047aec00efb admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:01.486654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424dcf29-085e-45ef-be65-9d84df6ce8ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:01.487315 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dcf29-085e-45ef-be65-9d84df6ce8ca service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:01.487597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dcf29-085e-45ef-be65-9d84df6ce8ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:01.487903 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dcf29-085e-45ef-be65-9d84df6ce8ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:01.549872 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424dcf29-085e-45ef-be65-9d84df6ce8ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:01.551330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424dcf29-085e-45ef-be65-9d84df6ce8ca service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:01.561204 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1038/2089] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:13:01 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:03.214835 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac26a9-4418-4ce9-9376-3a65c344d54b None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:03.215327 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac26a9-4418-4ce9-9376-3a65c344d54b None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:03.215453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac26a9-4418-4ce9-9376-3a65c344d54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:03.239623 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-dfac26a9-4418-4ce9-9376-3a65c344d54b None None] MFA Rules not processed for user `1cd36b543bd9487ebd8ce578e9a76c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:03.240959 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-dfac26a9-4418-4ce9-9376-3a65c344d54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:03.242609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424dcf29-085e-45ef-be65-9d84df6ce8ca service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:03.244065 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-424dcf29-085e-45ef-be65-9d84df6ce8ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:03.256428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:03.262135 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424dcf29-085e-45ef-be65-9d84df6ce8ca service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:03.268002 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35769437-396c-423e-b75b-b7ce96965e49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:03.268570 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35769437-396c-423e-b75b-b7ce96965e49 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:03.268862 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35769437-396c-423e-b75b-b7ce96965e49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:03.269169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35769437-396c-423e-b75b-b7ce96965e49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:03.319338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1052/2090] 10.222.0.100 () {58 vars in 1014 bytes} [Tue Aug 2 09:13:03 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:03.347087 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35769437-396c-423e-b75b-b7ce96965e49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:03.348633 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35769437-396c-423e-b75b-b7ce96965e49 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:03.359110 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1039/2091] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:13:03 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:13:03.374907 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac26a9-4418-4ce9-9376-3a65c344d54b None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:03.376593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-dfac26a9-4418-4ce9-9376-3a65c344d54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:03.459209 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac26a9-4418-4ce9-9376-3a65c344d54b None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:03.468328 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b4c283-2dfa-43b6-b517-53c15d171305 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50cb532b58ed4951ba960a0a7d579b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:03.469295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4c283-2dfa-43b6-b517-53c15d171305 service swift] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:03.469726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4c283-2dfa-43b6-b517-53c15d171305 service swift] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:03.470156 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4c283-2dfa-43b6-b517-53c15d171305 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:03.480116 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b4c283-2dfa-43b6-b517-53c15d171305 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:03.482942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b4c283-2dfa-43b6-b517-53c15d171305 service swift] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:03.493782 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1053/2092] 10.222.0.100 () {60 vars in 1389 bytes} [Tue Aug 2 09:13:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:13:04.369767 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35769437-396c-423e-b75b-b7ce96965e49 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:04.380055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:04.385254 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ac1ef-3b62-48a0-9fe8-2c57002c0e54 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:04.385817 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ac1ef-3b62-48a0-9fe8-2c57002c0e54 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:04.385817 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ac1ef-3b62-48a0-9fe8-2c57002c0e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:04.391529 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35769437-396c-423e-b75b-b7ce96965e49 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:04.407976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c4c93d-710d-4670-9277-a9fb03985f76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:04.408886 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4c93d-710d-4670-9277-a9fb03985f76 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:04.409179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4c93d-710d-4670-9277-a9fb03985f76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:04.409457 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4c93d-710d-4670-9277-a9fb03985f76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:04.416498 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-7e1ac1ef-3b62-48a0-9fe8-2c57002c0e54 None None] MFA Rules not processed for user `1cd36b543bd9487ebd8ce578e9a76c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:04.425662 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-7e1ac1ef-3b62-48a0-9fe8-2c57002c0e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:04.434736 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c4c93d-710d-4670-9277-a9fb03985f76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:04.440322 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c4c93d-710d-4670-9277-a9fb03985f76 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:04.441468 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1054/2093] 10.222.0.100 () {58 vars in 972 bytes} [Tue Aug 2 09:13:04 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:13:04.456710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ac1ef-3b62-48a0-9fe8-2c57002c0e54 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:04.459546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1040/2094] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:13:04 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:04.469966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ac1ef-3b62-48a0-9fe8-2c57002c0e54 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:04.475351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e925cb5-8c3b-466e-bf18-01fe07ffd24e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50cb532b58ed4951ba960a0a7d579b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:04.476089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e925cb5-8c3b-466e-bf18-01fe07ffd24e service swift] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:04.476439 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e925cb5-8c3b-466e-bf18-01fe07ffd24e service swift] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:04.476780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e925cb5-8c3b-466e-bf18-01fe07ffd24e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:04.486820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e925cb5-8c3b-466e-bf18-01fe07ffd24e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:04.488473 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e925cb5-8c3b-466e-bf18-01fe07ffd24e service swift] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:04.501163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1055/2095] 10.222.0.100 () {60 vars in 1389 bytes} [Tue Aug 2 09:13:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:13:04.786409 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65944ce-9ae9-4663-bbfb-7fd82c2599c8 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:04.786875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65944ce-9ae9-4663-bbfb-7fd82c2599c8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:04.787338 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65944ce-9ae9-4663-bbfb-7fd82c2599c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:04.809774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d65944ce-9ae9-4663-bbfb-7fd82c2599c8 None None] MFA Rules not processed for user `1cd36b543bd9487ebd8ce578e9a76c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:04.810938 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d65944ce-9ae9-4663-bbfb-7fd82c2599c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:04.819420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1041/2096] 10.222.0.100 () {58 vars in 1014 bytes} [Tue Aug 2 09:13:04 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:04.833053 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e925cb5-8c3b-466e-bf18-01fe07ffd24e service swift] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:04.844194 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e925cb5-8c3b-466e-bf18-01fe07ffd24e service swift] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:04.850170 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff05764-4a7e-4ecd-bda8-7a0edd64a1c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50cb532b58ed4951ba960a0a7d579b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:04.850826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff05764-4a7e-4ecd-bda8-7a0edd64a1c3 service swift] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:04.851132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff05764-4a7e-4ecd-bda8-7a0edd64a1c3 service swift] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:04.851506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff05764-4a7e-4ecd-bda8-7a0edd64a1c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:04.864133 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff05764-4a7e-4ecd-bda8-7a0edd64a1c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:04.865757 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff05764-4a7e-4ecd-bda8-7a0edd64a1c3 service swift] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:04.875562 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1056/2097] 10.222.0.100 () {60 vars in 1389 bytes} [Tue Aug 2 09:13:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:13:05.293886 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55ac43-41a0-4ce3-9604-96d356714654 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:05.294176 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55ac43-41a0-4ce3-9604-96d356714654 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:05.294452 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55ac43-41a0-4ce3-9604-96d356714654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:05.315416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-df55ac43-41a0-4ce3-9604-96d356714654 None None] MFA Rules not processed for user `1cd36b543bd9487ebd8ce578e9a76c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:05.316635 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-df55ac43-41a0-4ce3-9604-96d356714654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:05.324281 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1042/2098] 10.222.0.100 () {58 vars in 972 bytes} [Tue Aug 2 09:13:05 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:05.336949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff05764-4a7e-4ecd-bda8-7a0edd64a1c3 service swift] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:05.347503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff05764-4a7e-4ecd-bda8-7a0edd64a1c3 service swift] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:05.352596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e76e9-1ab8-451f-8865-668df044b9d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50cb532b58ed4951ba960a0a7d579b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:05.353276 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e76e9-1ab8-451f-8865-668df044b9d7 service swift] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:05.353557 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e76e9-1ab8-451f-8865-668df044b9d7 service swift] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:05.353828 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e76e9-1ab8-451f-8865-668df044b9d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:05.363077 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e76e9-1ab8-451f-8865-668df044b9d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:05.364650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e76e9-1ab8-451f-8865-668df044b9d7 service swift] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:05.373063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1057/2099] 10.222.0.100 () {60 vars in 1389 bytes} [Tue Aug 2 09:13:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:13:08.821899 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55ac43-41a0-4ce3-9604-96d356714654 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:08.825474 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-df55ac43-41a0-4ce3-9604-96d356714654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:08.960504 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55ac43-41a0-4ce3-9604-96d356714654 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:08.967368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987b2e6-1404-4fe3-857c-391749a194db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:08.969403 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987b2e6-1404-4fe3-857c-391749a194db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:08.969893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987b2e6-1404-4fe3-857c-391749a194db admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:08.970398 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987b2e6-1404-4fe3-857c-391749a194db admin admin] PATH_INFO: `/v3/users/14f3f9dd0bd0428a8143c6ef6568635c` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:08.972417 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987b2e6-1404-4fe3-857c-391749a194db admin admin] RBAC: Authorizing `identity:delete_user(user_id=14f3f9dd0bd0428a8143c6ef6568635c)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:08.975855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987b2e6-1404-4fe3-857c-391749a194db admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:09.044825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9987b2e6-1404-4fe3-857c-391749a194db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14f3f9dd0bd0428a8143c6ef6568635c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:09.053958 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9987b2e6-1404-4fe3-857c-391749a194db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14f3f9dd0bd0428a8143c6ef6568635c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:09.054729 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9987b2e6-1404-4fe3-857c-391749a194db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14f3f9dd0bd0428a8143c6ef6568635c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:09.065661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9987b2e6-1404-4fe3-857c-391749a194db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14f3f9dd0bd0428a8143c6ef6568635c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:09.066734 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9987b2e6-1404-4fe3-857c-391749a194db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14f3f9dd0bd0428a8143c6ef6568635c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:09.104382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1043/2100] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:13:08 2022] DELETE /identity/v3/users/14f3f9dd0bd0428a8143c6ef6568635c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:09.112843 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6eb956-2b58-4354-bf01-f14ad2214513 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:09.113449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6eb956-2b58-4354-bf01-f14ad2214513 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:09.113449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6eb956-2b58-4354-bf01-f14ad2214513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:09.155293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-dd6eb956-2b58-4354-bf01-f14ad2214513 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:09.156671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-dd6eb956-2b58-4354-bf01-f14ad2214513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:09.280924 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1058/2101] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:09.291028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987b2e6-1404-4fe3-857c-391749a194db admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:09.292826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9987b2e6-1404-4fe3-857c-391749a194db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:09.408239 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987b2e6-1404-4fe3-857c-391749a194db admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:09.412954 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337f3c1-2e70-4e31-8b28-305edbc1fe4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:09.413632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337f3c1-2e70-4e31-8b28-305edbc1fe4e service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:09.413976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337f3c1-2e70-4e31-8b28-305edbc1fe4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:09.414232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337f3c1-2e70-4e31-8b28-305edbc1fe4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:09.426545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337f3c1-2e70-4e31-8b28-305edbc1fe4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:09.428504 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337f3c1-2e70-4e31-8b28-305edbc1fe4e service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:09.435326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:09.444215 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1044/2102] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:13:09 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:09.742418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6eb956-2b58-4354-bf01-f14ad2214513 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:09.743952 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-dd6eb956-2b58-4354-bf01-f14ad2214513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:09.763395 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6eb956-2b58-4354-bf01-f14ad2214513 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:09.768786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6b398-d494-478d-a037-7662fc797b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:09.769516 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6b398-d494-478d-a037-7662fc797b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:09.769796 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6b398-d494-478d-a037-7662fc797b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:09.770071 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6b398-d494-478d-a037-7662fc797b33 admin admin] PATH_INFO: `/v3/projects/e250298a849145b9b7844caf5205c85d` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:09.771360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6b398-d494-478d-a037-7662fc797b33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e250298a849145b9b7844caf5205c85d)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:09.773348 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6b398-d494-478d-a037-7662fc797b33 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:09.775164 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:09.789269 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337f3c1-2e70-4e31-8b28-305edbc1fe4e service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:09.790679 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2337f3c1-2e70-4e31-8b28-305edbc1fe4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:09.810613 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337f3c1-2e70-4e31-8b28-305edbc1fe4e service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:09.816089 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9764cd0-c865-4ba7-93ab-bfa767557060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:09.816749 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9764cd0-c865-4ba7-93ab-bfa767557060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:09.817037 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9764cd0-c865-4ba7-93ab-bfa767557060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:09.817359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9764cd0-c865-4ba7-93ab-bfa767557060 admin admin] PATH_INFO: `/v3/users/f08fe9a161dd4757a70d4c2075436901` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:09.819132 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9764cd0-c865-4ba7-93ab-bfa767557060 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f08fe9a161dd4757a70d4c2075436901)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:09.822809 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9764cd0-c865-4ba7-93ab-bfa767557060 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:09.857056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-06e6b398-d494-478d-a037-7662fc797b33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e250298a849145b9b7844caf5205c85d', 'request_id': 'req-06e6b398-d494-478d-a037-7662fc797b33'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:09.871757 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-06e6b398-d494-478d-a037-7662fc797b33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e250298a849145b9b7844caf5205c85d', 'request_id': 'req-06e6b398-d494-478d-a037-7662fc797b33'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:09.887974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-06e6b398-d494-478d-a037-7662fc797b33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e250298a849145b9b7844caf5205c85d', 'request_id': 'req-06e6b398-d494-478d-a037-7662fc797b33'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:09.890249 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f9764cd0-c865-4ba7-93ab-bfa767557060 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f08fe9a161dd4757a70d4c2075436901'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:09.910121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-06e6b398-d494-478d-a037-7662fc797b33 admin admin] The token cache is being invalidate because project e250298a849145b9b7844caf5205c85d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:13:09.910633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-06e6b398-d494-478d-a037-7662fc797b33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:09.911396 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-06e6b398-d494-478d-a037-7662fc797b33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:09.913161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1059/2103] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:13:09 2022] DELETE /identity/v3/projects/e250298a849145b9b7844caf5205c85d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:13:09.918309 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10404c-9657-44f6-b603-326b17bd3fca None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:09.918982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10404c-9657-44f6-b603-326b17bd3fca None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:09.919443 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10404c-9657-44f6-b603-326b17bd3fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:09.920196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f9764cd0-c865-4ba7-93ab-bfa767557060 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f08fe9a161dd4757a70d4c2075436901'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:09.921288 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f9764cd0-c865-4ba7-93ab-bfa767557060 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f08fe9a161dd4757a70d4c2075436901'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:09.946668 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-7b10404c-9657-44f6-b603-326b17bd3fca None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:09.947298 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-7b10404c-9657-44f6-b603-326b17bd3fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:09.950010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f9764cd0-c865-4ba7-93ab-bfa767557060 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f08fe9a161dd4757a70d4c2075436901'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:09.950682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f9764cd0-c865-4ba7-93ab-bfa767557060 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f08fe9a161dd4757a70d4c2075436901'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:09.983403 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1045/2104] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:13:09 2022] DELETE /identity/v3/users/f08fe9a161dd4757a70d4c2075436901 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:13:09.988146 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a8bd2-b09d-4dee-bf87-312b7a4e9bb3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:09.988551 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a8bd2-b09d-4dee-bf87-312b7a4e9bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:09.988893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a8bd2-b09d-4dee-bf87-312b7a4e9bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:10.010217 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-4f4a8bd2-b09d-4dee-bf87-312b7a4e9bb3 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:10.011747 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4f4a8bd2-b09d-4dee-bf87-312b7a4e9bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:10.047951 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1060/2105] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:13:10.049671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f7934-0e67-4ab4-810c-e85e1905dad8 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:10.049971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f7934-0e67-4ab4-810c-e85e1905dad8 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:10.050254 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f7934-0e67-4ab4-810c-e85e1905dad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:10.081877 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-132f7934-0e67-4ab4-810c-e85e1905dad8 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:10.083099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-132f7934-0e67-4ab4-810c-e85e1905dad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:10.115720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1046/2106] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:13:10.129574 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a8bd2-b09d-4dee-bf87-312b7a4e9bb3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:10.131680 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4f4a8bd2-b09d-4dee-bf87-312b7a4e9bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:10.176086 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1061/2107] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:10.180365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f7934-0e67-4ab4-810c-e85e1905dad8 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:10.193264 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f7934-0e67-4ab4-810c-e85e1905dad8 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:10.199072 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daec8db-dea0-4741-bedf-8226dcdefe43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:10.199709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daec8db-dea0-4741-bedf-8226dcdefe43 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:10.199994 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daec8db-dea0-4741-bedf-8226dcdefe43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:10.200272 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daec8db-dea0-4741-bedf-8226dcdefe43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:10.200977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daec8db-dea0-4741-bedf-8226dcdefe43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:13:10.201939 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daec8db-dea0-4741-bedf-8226dcdefe43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:10.203893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daec8db-dea0-4741-bedf-8226dcdefe43 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:10.212103 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a8bd2-b09d-4dee-bf87-312b7a4e9bb3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:10.216172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0c252-d72b-42c8-9268-3ad8fd8938d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:10.216759 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1062/2108] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:13:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:13:10.216873 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0c252-d72b-42c8-9268-3ad8fd8938d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:10.217150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0c252-d72b-42c8-9268-3ad8fd8938d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:10.217867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0c252-d72b-42c8-9268-3ad8fd8938d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:10.221641 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daec8db-dea0-4741-bedf-8226dcdefe43 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:10.228075 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc0c252-d72b-42c8-9268-3ad8fd8938d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:10.229554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc0c252-d72b-42c8-9268-3ad8fd8938d7 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:10.232283 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daec8db-dea0-4741-bedf-8226dcdefe43 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:10.243088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1047/2109] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:13:10 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:10.245685 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c49130-8ab4-49df-9945-d1a2d47f1c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:10.246326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c49130-8ab4-49df-9945-d1a2d47f1c09 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:10.246610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c49130-8ab4-49df-9945-d1a2d47f1c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:10.246885 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c49130-8ab4-49df-9945-d1a2d47f1c09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:10.247562 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c49130-8ab4-49df-9945-d1a2d47f1c09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:10.250298 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c49130-8ab4-49df-9945-d1a2d47f1c09 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:10.269304 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:10.285217 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1063/2110] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:13:10 2022] POST /identity/v3/projects => generated 384 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:10.289752 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0c252-d72b-42c8-9268-3ad8fd8938d7 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:10.300491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0c252-d72b-42c8-9268-3ad8fd8938d7 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:10.315502 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:10.388299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213881b-d5f2-414e-b1ed-d2adaa2c98a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:10.388952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213881b-d5f2-414e-b1ed-d2adaa2c98a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:10.389943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213881b-d5f2-414e-b1ed-d2adaa2c98a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:10.390233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213881b-d5f2-414e-b1ed-d2adaa2c98a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:10.391086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213881b-d5f2-414e-b1ed-d2adaa2c98a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:10.393776 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213881b-d5f2-414e-b1ed-d2adaa2c98a3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:10.425647 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1048/2111] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:13:10 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:10.427564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c49130-8ab4-49df-9945-d1a2d47f1c09 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:10.441852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c49130-8ab4-49df-9945-d1a2d47f1c09 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:10.447734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb3b1a-a68f-4714-8e6c-80e1a8c9ee99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:10.448373 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb3b1a-a68f-4714-8e6c-80e1a8c9ee99 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:10.448661 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb3b1a-a68f-4714-8e6c-80e1a8c9ee99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:10.448941 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb3b1a-a68f-4714-8e6c-80e1a8c9ee99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:10.449700 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb3b1a-a68f-4714-8e6c-80e1a8c9ee99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:10.451372 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb3b1a-a68f-4714-8e6c-80e1a8c9ee99 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:10.473169 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1064/2112] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:13:10 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:10.473169 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb3b1a-a68f-4714-8e6c-80e1a8c9ee99 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:10.474758 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213881b-d5f2-414e-b1ed-d2adaa2c98a3 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:10.476754 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d213881b-d5f2-414e-b1ed-d2adaa2c98a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:10.489449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb3b1a-a68f-4714-8e6c-80e1a8c9ee99 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:10.492672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213881b-d5f2-414e-b1ed-d2adaa2c98a3 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:10.495606 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3be7b-c64a-4223-877b-b3b471f3508b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:10.496299 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3be7b-c64a-4223-877b-b3b471f3508b admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:10.496589 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3be7b-c64a-4223-877b-b3b471f3508b admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:10.497193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3be7b-c64a-4223-877b-b3b471f3508b admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:10.498144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3be7b-c64a-4223-877b-b3b471f3508b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:10.499791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8a315-0476-44b4-9000-10a3d8829b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:10.501502 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8a315-0476-44b4-9000-10a3d8829b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:10.501618 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3be7b-c64a-4223-877b-b3b471f3508b admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:10.502087 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8a315-0476-44b4-9000-10a3d8829b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:10.502563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8a315-0476-44b4-9000-10a3d8829b80 admin admin] PATH_INFO: `/v3/projects/11cb7b53383b4f708c9d508e90072d20` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:10.504576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8a315-0476-44b4-9000-10a3d8829b80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11cb7b53383b4f708c9d508e90072d20)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:10.507778 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8a315-0476-44b4-9000-10a3d8829b80 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:10.518261 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1065/2113] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:13:10 2022] GET /identity/v3/roles => generated 1427 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:10.522453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3be7b-c64a-4223-877b-b3b471f3508b admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:10.534118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3be7b-c64a-4223-877b-b3b471f3508b admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:10.539069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d3667-9237-4060-8180-63876cdf8b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:10.539757 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d3667-9237-4060-8180-63876cdf8b55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:10.540218 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d3667-9237-4060-8180-63876cdf8b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:10.540512 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d3667-9237-4060-8180-63876cdf8b55 admin admin] PATH_INFO: `/v3/projects/8a7c4ebdcb344adaa130bfa7f937111f/users/54b619ada5874d75882c48ccd006b680/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:10.562103 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d3667-9237-4060-8180-63876cdf8b55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7c4ebdcb344adaa130bfa7f937111f, user_id=54b619ada5874d75882c48ccd006b680, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:10.564958 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d3667-9237-4060-8180-63876cdf8b55 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:10.581745 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1066/2114] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:13:10 2022] PUT /identity/v3/projects/8a7c4ebdcb344adaa130bfa7f937111f/users/54b619ada5874d75882c48ccd006b680/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:10.594718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24789204-bf66-4318-b735-1770e25875cd None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:10.595028 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24789204-bf66-4318-b735-1770e25875cd None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:10.595280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24789204-bf66-4318-b735-1770e25875cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:10.596097 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8dc8a315-0476-44b4-9000-10a3d8829b80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11cb7b53383b4f708c9d508e90072d20', 'request_id': 'req-8dc8a315-0476-44b4-9000-10a3d8829b80'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:10.609886 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8dc8a315-0476-44b4-9000-10a3d8829b80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11cb7b53383b4f708c9d508e90072d20', 'request_id': 'req-8dc8a315-0476-44b4-9000-10a3d8829b80'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:10.623150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8dc8a315-0476-44b4-9000-10a3d8829b80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11cb7b53383b4f708c9d508e90072d20', 'request_id': 'req-8dc8a315-0476-44b4-9000-10a3d8829b80'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:10.631184 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-24789204-bf66-4318-b735-1770e25875cd None None] MFA Rules not processed for user `54b619ada5874d75882c48ccd006b680`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:10.632421 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-24789204-bf66-4318-b735-1770e25875cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:10.665903 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8dc8a315-0476-44b4-9000-10a3d8829b80 admin admin] The token cache is being invalidate because project 11cb7b53383b4f708c9d508e90072d20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:13:10.666300 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8dc8a315-0476-44b4-9000-10a3d8829b80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:10.667087 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8dc8a315-0476-44b4-9000-10a3d8829b80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:10.669810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1049/2115] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:13:10 2022] DELETE /identity/v3/projects/11cb7b53383b4f708c9d508e90072d20 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:10.756694 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1067/2116] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:13:10.778821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8a315-0476-44b4-9000-10a3d8829b80 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:10.781851 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8dc8a315-0476-44b4-9000-10a3d8829b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:10.867974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8a315-0476-44b4-9000-10a3d8829b80 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:10.932003 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:10.942174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40908fde-e98d-4715-bb51-51f6888efa78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:10.942826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40908fde-e98d-4715-bb51-51f6888efa78 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:10.943231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40908fde-e98d-4715-bb51-51f6888efa78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:10.943517 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40908fde-e98d-4715-bb51-51f6888efa78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:10.953705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40908fde-e98d-4715-bb51-51f6888efa78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:10.955151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40908fde-e98d-4715-bb51-51f6888efa78 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:10.965430 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1050/2117] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:13:10 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:11.989243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24789204-bf66-4318-b735-1770e25875cd None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:11.991304 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-24789204-bf66-4318-b735-1770e25875cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:12.061839 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24789204-bf66-4318-b735-1770e25875cd None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:12.067044 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527dd793-af85-4ab4-ba79-542ded0a3bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:12.067754 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527dd793-af85-4ab4-ba79-542ded0a3bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:12.068040 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527dd793-af85-4ab4-ba79-542ded0a3bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:12.068321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527dd793-af85-4ab4-ba79-542ded0a3bad admin admin] PATH_INFO: `/v3/users/d960d57ac8f743d798ca32a21b87cf16` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:12.069556 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527dd793-af85-4ab4-ba79-542ded0a3bad admin admin] RBAC: Authorizing `identity:delete_user(user_id=d960d57ac8f743d798ca32a21b87cf16)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:12.071578 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527dd793-af85-4ab4-ba79-542ded0a3bad admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:12.126530 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:12.135130 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-527dd793-af85-4ab4-ba79-542ded0a3bad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd960d57ac8f743d798ca32a21b87cf16'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:12.136404 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-527dd793-af85-4ab4-ba79-542ded0a3bad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd960d57ac8f743d798ca32a21b87cf16'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:12.148851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-527dd793-af85-4ab4-ba79-542ded0a3bad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd960d57ac8f743d798ca32a21b87cf16'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:12.167400 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-527dd793-af85-4ab4-ba79-542ded0a3bad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd960d57ac8f743d798ca32a21b87cf16'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:12.179478 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-527dd793-af85-4ab4-ba79-542ded0a3bad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd960d57ac8f743d798ca32a21b87cf16'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:12.187798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1068/2118] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:13:11 2022] DELETE /identity/v3/users/d960d57ac8f743d798ca32a21b87cf16 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:12.193284 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736df02-72db-430c-b889-ea4b3340e609 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:12.193842 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736df02-72db-430c-b889-ea4b3340e609 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:12.193929 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736df02-72db-430c-b889-ea4b3340e609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:12.221595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-7736df02-72db-430c-b889-ea4b3340e609 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:12.222747 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7736df02-72db-430c-b889-ea4b3340e609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:12.293868 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1051/2119] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:12.305170 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527dd793-af85-4ab4-ba79-542ded0a3bad admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:12.306639 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-527dd793-af85-4ab4-ba79-542ded0a3bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:12.374778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527dd793-af85-4ab4-ba79-542ded0a3bad admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:12.379943 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8401e7a-d34c-4fa8-b96f-70a1a2f54733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:12.381135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8401e7a-d34c-4fa8-b96f-70a1a2f54733 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:12.381520 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8401e7a-d34c-4fa8-b96f-70a1a2f54733 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:12.381894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8401e7a-d34c-4fa8-b96f-70a1a2f54733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:12.393673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8401e7a-d34c-4fa8-b96f-70a1a2f54733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:12.395415 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8401e7a-d34c-4fa8-b96f-70a1a2f54733 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:12.407947 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1069/2120] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:13:12 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:13.110448 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736df02-72db-430c-b889-ea4b3340e609 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:13.112113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7736df02-72db-430c-b889-ea4b3340e609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:13.127056 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736df02-72db-430c-b889-ea4b3340e609 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:13.133933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5988bf-e457-4e38-bcf2-864e71360e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:13.135021 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5988bf-e457-4e38-bcf2-864e71360e89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:13.135692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5988bf-e457-4e38-bcf2-864e71360e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:13.136043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5988bf-e457-4e38-bcf2-864e71360e89 admin admin] PATH_INFO: `/v3/projects/62aa4e761f5f4a2f8adf0d75d587ca62` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:13.137777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5988bf-e457-4e38-bcf2-864e71360e89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62aa4e761f5f4a2f8adf0d75d587ca62)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:13.143186 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5988bf-e457-4e38-bcf2-864e71360e89 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:13.243390 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4e5988bf-e457-4e38-bcf2-864e71360e89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62aa4e761f5f4a2f8adf0d75d587ca62', 'request_id': 'req-4e5988bf-e457-4e38-bcf2-864e71360e89'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:13.252154 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4e5988bf-e457-4e38-bcf2-864e71360e89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62aa4e761f5f4a2f8adf0d75d587ca62', 'request_id': 'req-4e5988bf-e457-4e38-bcf2-864e71360e89'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:13.261961 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4e5988bf-e457-4e38-bcf2-864e71360e89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62aa4e761f5f4a2f8adf0d75d587ca62', 'request_id': 'req-4e5988bf-e457-4e38-bcf2-864e71360e89'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:13.280038 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4e5988bf-e457-4e38-bcf2-864e71360e89 admin admin] The token cache is being invalidate because project 62aa4e761f5f4a2f8adf0d75d587ca62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:13:13.281040 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4e5988bf-e457-4e38-bcf2-864e71360e89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:13.281906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4e5988bf-e457-4e38-bcf2-864e71360e89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:13.283770 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1052/2121] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:13:13 2022] DELETE /identity/v3/projects/62aa4e761f5f4a2f8adf0d75d587ca62 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:13.288875 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8401e7a-d34c-4fa8-b96f-70a1a2f54733 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:13.308061 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8401e7a-d34c-4fa8-b96f-70a1a2f54733 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:13.376923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a086842-e0b8-48b5-a85e-4d68e638942a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:13.377609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a086842-e0b8-48b5-a85e-4d68e638942a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:13.377894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a086842-e0b8-48b5-a85e-4d68e638942a admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:13.378170 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a086842-e0b8-48b5-a85e-4d68e638942a admin admin] PATH_INFO: `/v3/users/6d291903696749828f70df314ecbe6e5` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:13.379380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a086842-e0b8-48b5-a85e-4d68e638942a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d291903696749828f70df314ecbe6e5)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:13.381611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a086842-e0b8-48b5-a85e-4d68e638942a admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:13.437450 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-1a086842-e0b8-48b5-a85e-4d68e638942a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d291903696749828f70df314ecbe6e5'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:13.438119 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-1a086842-e0b8-48b5-a85e-4d68e638942a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d291903696749828f70df314ecbe6e5'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:13.446260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-1a086842-e0b8-48b5-a85e-4d68e638942a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d291903696749828f70df314ecbe6e5'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:13.453952 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:13.460785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-1a086842-e0b8-48b5-a85e-4d68e638942a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d291903696749828f70df314ecbe6e5'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:13.472161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-1a086842-e0b8-48b5-a85e-4d68e638942a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d291903696749828f70df314ecbe6e5'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:13.479501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1070/2122] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:13:13 2022] DELETE /identity/v3/users/6d291903696749828f70df314ecbe6e5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:13.590304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5988bf-e457-4e38-bcf2-864e71360e89 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:13.591790 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4e5988bf-e457-4e38-bcf2-864e71360e89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:13.608575 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5988bf-e457-4e38-bcf2-864e71360e89 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:13.704361 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5f636-56e6-43f5-866d-5275b8058568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:13.705340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5f636-56e6-43f5-866d-5275b8058568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:13.705633 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5f636-56e6-43f5-866d-5275b8058568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:13.705992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5f636-56e6-43f5-866d-5275b8058568 admin admin] PATH_INFO: `/v3/users/fd776c3e90a94dcf94faa55235e9cb85` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:13.707388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5f636-56e6-43f5-866d-5275b8058568 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd776c3e90a94dcf94faa55235e9cb85)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:13.709605 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5f636-56e6-43f5-866d-5275b8058568 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:13.729991 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:13.813584 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-67e5f636-56e6-43f5-866d-5275b8058568 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd776c3e90a94dcf94faa55235e9cb85'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:13.819415 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a086842-e0b8-48b5-a85e-4d68e638942a admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:13.821387 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1a086842-e0b8-48b5-a85e-4d68e638942a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:13.822667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-67e5f636-56e6-43f5-866d-5275b8058568 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd776c3e90a94dcf94faa55235e9cb85'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:13.824125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-67e5f636-56e6-43f5-866d-5275b8058568 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd776c3e90a94dcf94faa55235e9cb85'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:13.833368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-67e5f636-56e6-43f5-866d-5275b8058568 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd776c3e90a94dcf94faa55235e9cb85'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:13.834167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-67e5f636-56e6-43f5-866d-5275b8058568 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd776c3e90a94dcf94faa55235e9cb85'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:13.847754 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a086842-e0b8-48b5-a85e-4d68e638942a admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:13.854311 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66decc27-f667-4a15-b94f-211c267e59b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:13.855143 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66decc27-f667-4a15-b94f-211c267e59b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:13.855485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66decc27-f667-4a15-b94f-211c267e59b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:13.855766 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66decc27-f667-4a15-b94f-211c267e59b8 admin admin] PATH_INFO: `/v3/users/54b619ada5874d75882c48ccd006b680` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:13.857349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66decc27-f667-4a15-b94f-211c267e59b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54b619ada5874d75882c48ccd006b680)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:13.860325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66decc27-f667-4a15-b94f-211c267e59b8 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:13.861062 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1053/2123] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:13:13 2022] DELETE /identity/v3/users/fd776c3e90a94dcf94faa55235e9cb85 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:13:13.865946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1df17e-8c1d-4099-a751-ec8f445a68b1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:13.866256 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1df17e-8c1d-4099-a751-ec8f445a68b1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:13.866521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1df17e-8c1d-4099-a751-ec8f445a68b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:13.889977 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-bb1df17e-8c1d-4099-a751-ec8f445a68b1 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:13.891416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bb1df17e-8c1d-4099-a751-ec8f445a68b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:13.954992 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-66decc27-f667-4a15-b94f-211c267e59b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54b619ada5874d75882c48ccd006b680'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:13.956721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-66decc27-f667-4a15-b94f-211c267e59b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54b619ada5874d75882c48ccd006b680'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:13.965689 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-66decc27-f667-4a15-b94f-211c267e59b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54b619ada5874d75882c48ccd006b680'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:13.984951 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-66decc27-f667-4a15-b94f-211c267e59b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54b619ada5874d75882c48ccd006b680'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:13.993585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1054/2124] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:13.998804 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-66decc27-f667-4a15-b94f-211c267e59b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54b619ada5874d75882c48ccd006b680'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:14.001632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1df17e-8c1d-4099-a751-ec8f445a68b1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:14.003316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bb1df17e-8c1d-4099-a751-ec8f445a68b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:14.015905 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1071/2125] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:13:13 2022] DELETE /identity/v3/users/54b619ada5874d75882c48ccd006b680 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:13:14.017897 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516c78a-6e8e-422b-9f25-a8984dc7b76e None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:14.018357 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516c78a-6e8e-422b-9f25-a8984dc7b76e None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:14.018735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516c78a-6e8e-422b-9f25-a8984dc7b76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:14.045025 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-2516c78a-6e8e-422b-9f25-a8984dc7b76e None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:14.048083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-2516c78a-6e8e-422b-9f25-a8984dc7b76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:14.101416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1df17e-8c1d-4099-a751-ec8f445a68b1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:14.153424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1072/2126] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:14.155829 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516c78a-6e8e-422b-9f25-a8984dc7b76e None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:14.157537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-2516c78a-6e8e-422b-9f25-a8984dc7b76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:14.162487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:14.174388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35edb5d-1634-4f58-8545-00e67a8e7948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:14.175060 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35edb5d-1634-4f58-8545-00e67a8e7948 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:14.175342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35edb5d-1634-4f58-8545-00e67a8e7948 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:14.175616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35edb5d-1634-4f58-8545-00e67a8e7948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:14.179154 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f35edb5d-1634-4f58-8545-00e67a8e7948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:14.179289 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516c78a-6e8e-422b-9f25-a8984dc7b76e None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:14.193619 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1ba45-9568-4556-8cf3-e90cbc11685d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:14.194700 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1ba45-9568-4556-8cf3-e90cbc11685d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:14.195211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1ba45-9568-4556-8cf3-e90cbc11685d admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:14.195747 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1ba45-9568-4556-8cf3-e90cbc11685d admin admin] PATH_INFO: `/v3/projects/c2c133b1e6874f47828cf55bc2aedd9c` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:14.198371 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1ba45-9568-4556-8cf3-e90cbc11685d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2c133b1e6874f47828cf55bc2aedd9c)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:14.200834 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1ba45-9568-4556-8cf3-e90cbc11685d admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:14.226996 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35edb5d-1634-4f58-8545-00e67a8e7948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:14.228782 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35edb5d-1634-4f58-8545-00e67a8e7948 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:14.243399 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1055/2127] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:13:14 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:13:14.244358 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35edb5d-1634-4f58-8545-00e67a8e7948 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:14.246197 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f35edb5d-1634-4f58-8545-00e67a8e7948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:14.260902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35edb5d-1634-4f58-8545-00e67a8e7948 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:14.281610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:14.283211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-80e1ba45-9568-4556-8cf3-e90cbc11685d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2c133b1e6874f47828cf55bc2aedd9c', 'request_id': 'req-80e1ba45-9568-4556-8cf3-e90cbc11685d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:14.315474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-80e1ba45-9568-4556-8cf3-e90cbc11685d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2c133b1e6874f47828cf55bc2aedd9c', 'request_id': 'req-80e1ba45-9568-4556-8cf3-e90cbc11685d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:14.332575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-80e1ba45-9568-4556-8cf3-e90cbc11685d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2c133b1e6874f47828cf55bc2aedd9c', 'request_id': 'req-80e1ba45-9568-4556-8cf3-e90cbc11685d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:14.332751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c20c0-ac11-4317-928d-9df4f2a18472 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:14.333460 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c20c0-ac11-4317-928d-9df4f2a18472 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:14.335317 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c20c0-ac11-4317-928d-9df4f2a18472 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:14.335628 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c20c0-ac11-4317-928d-9df4f2a18472 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:14.362657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-80e1ba45-9568-4556-8cf3-e90cbc11685d admin admin] The token cache is being invalidate because project c2c133b1e6874f47828cf55bc2aedd9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:13:14.363056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-80e1ba45-9568-4556-8cf3-e90cbc11685d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:14.363913 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-80e1ba45-9568-4556-8cf3-e90cbc11685d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:14.368963 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1073/2128] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:13:14 2022] DELETE /identity/v3/projects/c2c133b1e6874f47828cf55bc2aedd9c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:14.371015 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1ba45-9568-4556-8cf3-e90cbc11685d admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:14.373168 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-80e1ba45-9568-4556-8cf3-e90cbc11685d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:14.428767 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c20c0-ac11-4317-928d-9df4f2a18472 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:14.430286 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c20c0-ac11-4317-928d-9df4f2a18472 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:14.445628 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1056/2129] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:13:14 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:14.487823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1ba45-9568-4556-8cf3-e90cbc11685d admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:14.494475 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb05a4-4cf3-4e23-b334-3f874e0ecc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:14.495246 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb05a4-4cf3-4e23-b334-3f874e0ecc00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:14.495778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb05a4-4cf3-4e23-b334-3f874e0ecc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:14.496088 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb05a4-4cf3-4e23-b334-3f874e0ecc00 admin admin] PATH_INFO: `/v3/users/63522869a31746ffa2e6aed07af37a2c` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:14.497432 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb05a4-4cf3-4e23-b334-3f874e0ecc00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63522869a31746ffa2e6aed07af37a2c)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:14.500456 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb05a4-4cf3-4e23-b334-3f874e0ecc00 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:14.589080 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-32eb05a4-4cf3-4e23-b334-3f874e0ecc00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63522869a31746ffa2e6aed07af37a2c'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:14.590334 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-32eb05a4-4cf3-4e23-b334-3f874e0ecc00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63522869a31746ffa2e6aed07af37a2c'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:14.598971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-32eb05a4-4cf3-4e23-b334-3f874e0ecc00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63522869a31746ffa2e6aed07af37a2c'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:14.622865 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-32eb05a4-4cf3-4e23-b334-3f874e0ecc00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63522869a31746ffa2e6aed07af37a2c'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:14.633566 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-32eb05a4-4cf3-4e23-b334-3f874e0ecc00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63522869a31746ffa2e6aed07af37a2c'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:14.641533 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1074/2130] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:13:14 2022] DELETE /identity/v3/users/63522869a31746ffa2e6aed07af37a2c => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:13:15.160154 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c20c0-ac11-4317-928d-9df4f2a18472 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:15.168721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb05a4-4cf3-4e23-b334-3f874e0ecc00 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:15.172576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-32eb05a4-4cf3-4e23-b334-3f874e0ecc00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:15.223659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c20c0-ac11-4317-928d-9df4f2a18472 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:15.311235 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a6349-5197-4ab6-a7f1-f2ab222540ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:15.313028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a6349-5197-4ab6-a7f1-f2ab222540ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:15.313028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a6349-5197-4ab6-a7f1-f2ab222540ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:15.313028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a6349-5197-4ab6-a7f1-f2ab222540ca admin admin] PATH_INFO: `/v3/projects/3a2df9e33aa043f4816b2d15fa76ad86` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:15.314008 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a6349-5197-4ab6-a7f1-f2ab222540ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a2df9e33aa043f4816b2d15fa76ad86)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:15.316115 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a6349-5197-4ab6-a7f1-f2ab222540ca admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:15.373035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:15.392577 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb05a4-4cf3-4e23-b334-3f874e0ecc00 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:15.398945 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:15.410014 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-070a6349-5197-4ab6-a7f1-f2ab222540ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a2df9e33aa043f4816b2d15fa76ad86', 'request_id': 'req-070a6349-5197-4ab6-a7f1-f2ab222540ca'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:15.419775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-070a6349-5197-4ab6-a7f1-f2ab222540ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a2df9e33aa043f4816b2d15fa76ad86', 'request_id': 'req-070a6349-5197-4ab6-a7f1-f2ab222540ca'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:15.431090 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-070a6349-5197-4ab6-a7f1-f2ab222540ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a2df9e33aa043f4816b2d15fa76ad86', 'request_id': 'req-070a6349-5197-4ab6-a7f1-f2ab222540ca'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:15.447969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-070a6349-5197-4ab6-a7f1-f2ab222540ca admin admin] The token cache is being invalidate because project 3a2df9e33aa043f4816b2d15fa76ad86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:13:15.448742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-070a6349-5197-4ab6-a7f1-f2ab222540ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:15.449750 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-070a6349-5197-4ab6-a7f1-f2ab222540ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:15.451698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1057/2131] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:13:15 2022] DELETE /identity/v3/projects/3a2df9e33aa043f4816b2d15fa76ad86 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:15.452502 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a6349-5197-4ab6-a7f1-f2ab222540ca admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:15.454339 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-070a6349-5197-4ab6-a7f1-f2ab222540ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:15.529491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0269cec-a66b-407a-82fa-d6db2fa4c239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:15.530703 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0269cec-a66b-407a-82fa-d6db2fa4c239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:15.531981 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0269cec-a66b-407a-82fa-d6db2fa4c239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:15.532169 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0269cec-a66b-407a-82fa-d6db2fa4c239 admin admin] PATH_INFO: `/v3/projects/d1d95a5c8c834b9f8f1e20e9990b54df` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:15.533578 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0269cec-a66b-407a-82fa-d6db2fa4c239 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1d95a5c8c834b9f8f1e20e9990b54df)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:15.535548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0269cec-a66b-407a-82fa-d6db2fa4c239 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:15.591938 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a6349-5197-4ab6-a7f1-f2ab222540ca admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:15.615125 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-a0269cec-a66b-407a-82fa-d6db2fa4c239 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1d95a5c8c834b9f8f1e20e9990b54df', 'request_id': 'req-a0269cec-a66b-407a-82fa-d6db2fa4c239'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:15.623657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-a0269cec-a66b-407a-82fa-d6db2fa4c239 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1d95a5c8c834b9f8f1e20e9990b54df', 'request_id': 'req-a0269cec-a66b-407a-82fa-d6db2fa4c239'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:15.637979 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-a0269cec-a66b-407a-82fa-d6db2fa4c239 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1d95a5c8c834b9f8f1e20e9990b54df', 'request_id': 'req-a0269cec-a66b-407a-82fa-d6db2fa4c239'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:15.660312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-a0269cec-a66b-407a-82fa-d6db2fa4c239 admin admin] The token cache is being invalidate because project d1d95a5c8c834b9f8f1e20e9990b54df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:13:15.660709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-a0269cec-a66b-407a-82fa-d6db2fa4c239 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:15.661547 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-a0269cec-a66b-407a-82fa-d6db2fa4c239 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:15.666609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1075/2132] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:13:15 2022] DELETE /identity/v3/projects/d1d95a5c8c834b9f8f1e20e9990b54df => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:15.666609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94885036-65d5-450f-a5d4-ee8eb9fe9dac None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:15.666609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94885036-65d5-450f-a5d4-ee8eb9fe9dac None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:15.667846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94885036-65d5-450f-a5d4-ee8eb9fe9dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:15.693751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f5670-a1fe-4598-a7d7-1ce7032e867e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:15.694470 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f5670-a1fe-4598-a7d7-1ce7032e867e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:15.694759 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f5670-a1fe-4598-a7d7-1ce7032e867e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:15.695042 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f5670-a1fe-4598-a7d7-1ce7032e867e admin admin] PATH_INFO: `/v3/projects/8a7c4ebdcb344adaa130bfa7f937111f` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:15.696533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f5670-a1fe-4598-a7d7-1ce7032e867e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a7c4ebdcb344adaa130bfa7f937111f)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:15.698546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f5670-a1fe-4598-a7d7-1ce7032e867e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:15.704304 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-94885036-65d5-450f-a5d4-ee8eb9fe9dac None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:15.705672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-94885036-65d5-450f-a5d4-ee8eb9fe9dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:15.723356 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1076/2133] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:15.726068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4732f-aaca-472b-bbda-24fa169d7eb2 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:15.726373 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4732f-aaca-472b-bbda-24fa169d7eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:15.726795 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4732f-aaca-472b-bbda-24fa169d7eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:15.760758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-69e4732f-aaca-472b-bbda-24fa169d7eb2 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:15.762140 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-69e4732f-aaca-472b-bbda-24fa169d7eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:15.769632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-020f5670-a1fe-4598-a7d7-1ce7032e867e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a7c4ebdcb344adaa130bfa7f937111f', 'request_id': 'req-020f5670-a1fe-4598-a7d7-1ce7032e867e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:15.777938 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-020f5670-a1fe-4598-a7d7-1ce7032e867e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a7c4ebdcb344adaa130bfa7f937111f', 'request_id': 'req-020f5670-a1fe-4598-a7d7-1ce7032e867e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:15.790400 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-020f5670-a1fe-4598-a7d7-1ce7032e867e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a7c4ebdcb344adaa130bfa7f937111f', 'request_id': 'req-020f5670-a1fe-4598-a7d7-1ce7032e867e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:15.803253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-020f5670-a1fe-4598-a7d7-1ce7032e867e admin admin] The token cache is being invalidate because project 8a7c4ebdcb344adaa130bfa7f937111f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:13:15.803543 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-020f5670-a1fe-4598-a7d7-1ce7032e867e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:15.804224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-020f5670-a1fe-4598-a7d7-1ce7032e867e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:15.805875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1058/2134] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:13:15 2022] DELETE /identity/v3/projects/8a7c4ebdcb344adaa130bfa7f937111f => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:15.807590 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5846c52-e796-4567-8969-6233ceb9cc85 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:15.807881 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5846c52-e796-4567-8969-6233ceb9cc85 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:15.808155 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5846c52-e796-4567-8969-6233ceb9cc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:15.824231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:15.833681 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d5846c52-e796-4567-8969-6233ceb9cc85 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:15.836328 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d5846c52-e796-4567-8969-6233ceb9cc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:15.856912 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1077/2135] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:15.857685 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0748e93-2a07-4da3-87ce-04135cccfe85 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:15.858490 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0748e93-2a07-4da3-87ce-04135cccfe85 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:15.859270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0748e93-2a07-4da3-87ce-04135cccfe85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:15.882892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-d0748e93-2a07-4da3-87ce-04135cccfe85 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:15.883345 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d0748e93-2a07-4da3-87ce-04135cccfe85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:15.901546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1078/2136] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:15.901546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebcbc3-9b0e-47eb-89b3-8521855b2bf3 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:15.901546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebcbc3-9b0e-47eb-89b3-8521855b2bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:15.901546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebcbc3-9b0e-47eb-89b3-8521855b2bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:15.920293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1059/2137] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:15.922429 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c9aeb-8372-4f9e-bbb9-5a177fab2c54 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:15.922637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c9aeb-8372-4f9e-bbb9-5a177fab2c54 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:15.922913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c9aeb-8372-4f9e-bbb9-5a177fab2c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:15.927002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-23ebcbc3-9b0e-47eb-89b3-8521855b2bf3 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:15.927002 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-23ebcbc3-9b0e-47eb-89b3-8521855b2bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:15.944808 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1079/2138] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:15.944808 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebcbc3-9b0e-47eb-89b3-8521855b2bf3 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:15.946753 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-890c9aeb-8372-4f9e-bbb9-5a177fab2c54 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:15.947994 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-890c9aeb-8372-4f9e-bbb9-5a177fab2c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:15.952636 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebcbc3-9b0e-47eb-89b3-8521855b2bf3 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:15.956686 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1060/2139] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:15.957558 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c9aeb-8372-4f9e-bbb9-5a177fab2c54 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:15.957876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cc287-88a2-457b-a1d1-158891e45c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:15.959396 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cc287-88a2-457b-a1d1-158891e45c0e admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:15.959885 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cc287-88a2-457b-a1d1-158891e45c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:15.960317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cc287-88a2-457b-a1d1-158891e45c0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:15.961436 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0cc287-88a2-457b-a1d1-158891e45c0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:13:15.961906 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0cc287-88a2-457b-a1d1-158891e45c0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:15.964606 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0cc287-88a2-457b-a1d1-158891e45c0e admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:15.968201 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c9aeb-8372-4f9e-bbb9-5a177fab2c54 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:15.973954 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56521b-f9be-446d-a2da-857504f6a47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:15.974677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56521b-f9be-446d-a2da-857504f6a47a admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:15.975082 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56521b-f9be-446d-a2da-857504f6a47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:15.975421 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56521b-f9be-446d-a2da-857504f6a47a admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:15.976135 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56521b-f9be-446d-a2da-857504f6a47a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:13:15.976440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56521b-f9be-446d-a2da-857504f6a47a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:15.976793 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:15.978226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56521b-f9be-446d-a2da-857504f6a47a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:15.982120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1080/2140] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:13:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:13:15.983931 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cc287-88a2-457b-a1d1-158891e45c0e admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:15.999818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1061/2141] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:13:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:13:15.999818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56521b-f9be-446d-a2da-857504f6a47a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:16.003071 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cc287-88a2-457b-a1d1-158891e45c0e admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:16.009426 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8fd02-301a-43e6-9495-0d5fda48f061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:16.010291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8fd02-301a-43e6-9495-0d5fda48f061 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:16.010502 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56521b-f9be-446d-a2da-857504f6a47a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:16.010763 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8fd02-301a-43e6-9495-0d5fda48f061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:16.011224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8fd02-301a-43e6-9495-0d5fda48f061 admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:16.012091 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8fd02-301a-43e6-9495-0d5fda48f061 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:13:16.012467 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8fd02-301a-43e6-9495-0d5fda48f061 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:16.014203 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8fd02-301a-43e6-9495-0d5fda48f061 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:16.015485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63120e9a-a759-4846-8388-ee70a8865518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:16.016341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63120e9a-a759-4846-8388-ee70a8865518 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:16.016448 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63120e9a-a759-4846-8388-ee70a8865518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:16.016685 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63120e9a-a759-4846-8388-ee70a8865518 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:16.017920 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63120e9a-a759-4846-8388-ee70a8865518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:16.019377 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63120e9a-a759-4846-8388-ee70a8865518 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:16.033263 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1081/2142] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:13:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:13:16.034184 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8fd02-301a-43e6-9495-0d5fda48f061 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:16.048941 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8fd02-301a-43e6-9495-0d5fda48f061 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:16.054414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593a91e-ad50-4c20-bb81-06f54c18a506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:16.055109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593a91e-ad50-4c20-bb81-06f54c18a506 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:16.055338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593a91e-ad50-4c20-bb81-06f54c18a506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:16.055604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593a91e-ad50-4c20-bb81-06f54c18a506 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:16.056645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593a91e-ad50-4c20-bb81-06f54c18a506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:16.056797 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1062/2143] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:13:15 2022] POST /identity/v3/projects => generated 376 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:16.057441 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63120e9a-a759-4846-8388-ee70a8865518 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:16.058959 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593a91e-ad50-4c20-bb81-06f54c18a506 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:16.094067 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1082/2144] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:13:16 2022] POST /identity/v3/projects => generated 372 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:16.095234 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593a91e-ad50-4c20-bb81-06f54c18a506 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:16.136559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63120e9a-a759-4846-8388-ee70a8865518 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:16.163125 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593a91e-ad50-4c20-bb81-06f54c18a506 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:16.167977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7dbe23-fadb-4dc2-8f6e-89e6af4a4884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:16.168611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7dbe23-fadb-4dc2-8f6e-89e6af4a4884 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:16.169036 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7dbe23-fadb-4dc2-8f6e-89e6af4a4884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:16.169356 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7dbe23-fadb-4dc2-8f6e-89e6af4a4884 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:16.170032 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7dbe23-fadb-4dc2-8f6e-89e6af4a4884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:16.172546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7dbe23-fadb-4dc2-8f6e-89e6af4a4884 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:16.198047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1083/2145] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:13:16 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:16.198872 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7dbe23-fadb-4dc2-8f6e-89e6af4a4884 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:16.203869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d585c7-c6e9-4de4-94cc-31c80c0ebc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:16.204539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d585c7-c6e9-4de4-94cc-31c80c0ebc0b admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:16.204834 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d585c7-c6e9-4de4-94cc-31c80c0ebc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:16.205140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d585c7-c6e9-4de4-94cc-31c80c0ebc0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:16.205820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d585c7-c6e9-4de4-94cc-31c80c0ebc0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:16.207763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d585c7-c6e9-4de4-94cc-31c80c0ebc0b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:16.209419 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7dbe23-fadb-4dc2-8f6e-89e6af4a4884 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:16.214056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6a6e0-c5ec-4bd5-9c72-a513f58ab573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:16.214676 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6a6e0-c5ec-4bd5-9c72-a513f58ab573 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:16.214960 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6a6e0-c5ec-4bd5-9c72-a513f58ab573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:16.215234 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6a6e0-c5ec-4bd5-9c72-a513f58ab573 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:16.215903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6a6e0-c5ec-4bd5-9c72-a513f58ab573 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:16.218494 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6a6e0-c5ec-4bd5-9c72-a513f58ab573 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:16.232456 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:16.248393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1084/2146] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:13:16 2022] POST /identity/v3/users => generated 287 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:16.250131 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6a6e0-c5ec-4bd5-9c72-a513f58ab573 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:16.252230 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1063/2147] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:13:16 2022] POST /identity/v3/projects => generated 374 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:16.253235 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d585c7-c6e9-4de4-94cc-31c80c0ebc0b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:16.263192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6a6e0-c5ec-4bd5-9c72-a513f58ab573 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:16.323028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d585c7-c6e9-4de4-94cc-31c80c0ebc0b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:16.329355 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99545247-0699-4a34-b6ba-5fda97b487a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:16.330012 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99545247-0699-4a34-b6ba-5fda97b487a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:16.330326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99545247-0699-4a34-b6ba-5fda97b487a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:16.330603 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99545247-0699-4a34-b6ba-5fda97b487a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:16.331309 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99545247-0699-4a34-b6ba-5fda97b487a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:16.332927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99545247-0699-4a34-b6ba-5fda97b487a1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:16.333507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01769a07-019b-465d-9626-6540ceb3cbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:16.334154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01769a07-019b-465d-9626-6540ceb3cbdf admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:16.334423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01769a07-019b-465d-9626-6540ceb3cbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:16.334694 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01769a07-019b-465d-9626-6540ceb3cbdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:16.335394 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01769a07-019b-465d-9626-6540ceb3cbdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:16.336990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01769a07-019b-465d-9626-6540ceb3cbdf admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:16.344317 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1064/2148] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:13:16 2022] GET /identity/v3/roles => generated 1427 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:16.345437 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99545247-0699-4a34-b6ba-5fda97b487a1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:16.351324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1085/2149] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:13:16 2022] GET /identity/v3/roles => generated 1427 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:16.351324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01769a07-019b-465d-9626-6540ceb3cbdf admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:16.357048 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99545247-0699-4a34-b6ba-5fda97b487a1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:16.359239 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01769a07-019b-465d-9626-6540ceb3cbdf admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:16.362697 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b092277-d506-4063-b77e-932459f53614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:16.363442 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b092277-d506-4063-b77e-932459f53614 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:16.363728 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b092277-d506-4063-b77e-932459f53614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:16.364013 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b092277-d506-4063-b77e-932459f53614 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:16.364177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc7cab-98eb-4f18-98bb-6d02bf1a072f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:16.364852 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b092277-d506-4063-b77e-932459f53614 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:16.364977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc7cab-98eb-4f18-98bb-6d02bf1a072f admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:16.365276 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc7cab-98eb-4f18-98bb-6d02bf1a072f admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:16.365520 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc7cab-98eb-4f18-98bb-6d02bf1a072f admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:16.366211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc7cab-98eb-4f18-98bb-6d02bf1a072f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:16.366816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b092277-d506-4063-b77e-932459f53614 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:16.367800 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc7cab-98eb-4f18-98bb-6d02bf1a072f admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:16.378887 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1086/2150] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:13:16 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:16.379524 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc7cab-98eb-4f18-98bb-6d02bf1a072f admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:16.391296 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc7cab-98eb-4f18-98bb-6d02bf1a072f admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:16.396144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72af1fb-f364-4b54-b22b-dc52c1894161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:16.396764 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72af1fb-f364-4b54-b22b-dc52c1894161 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:16.397042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72af1fb-f364-4b54-b22b-dc52c1894161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:16.397346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72af1fb-f364-4b54-b22b-dc52c1894161 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:16.400674 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1065/2151] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:13:16 2022] POST /identity/v3/users => generated 288 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:16.400853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72af1fb-f364-4b54-b22b-dc52c1894161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:16.400953 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b092277-d506-4063-b77e-932459f53614 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:16.401057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72af1fb-f364-4b54-b22b-dc52c1894161 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:16.410818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b092277-d506-4063-b77e-932459f53614 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:16.411440 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1087/2152] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:13:16 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:16.412243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72af1fb-f364-4b54-b22b-dc52c1894161 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:16.419598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41360e-6736-433a-ad16-94cdef928b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:16.420335 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41360e-6736-433a-ad16-94cdef928b49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:16.420698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41360e-6736-433a-ad16-94cdef928b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:16.421270 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41360e-6736-433a-ad16-94cdef928b49 admin admin] PATH_INFO: `/v3/projects/a6dd8f336ea44f14bfaf1c127ddbacb1/users/4f1e5fb6488d4478b8f4449295613e50/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:16.425327 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72af1fb-f364-4b54-b22b-dc52c1894161 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:16.437242 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65dd0d-afb0-437c-b694-dc657fbb9e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:16.437493 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41360e-6736-433a-ad16-94cdef928b49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6dd8f336ea44f14bfaf1c127ddbacb1, user_id=4f1e5fb6488d4478b8f4449295613e50, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:16.437977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65dd0d-afb0-437c-b694-dc657fbb9e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:16.438263 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65dd0d-afb0-437c-b694-dc657fbb9e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:16.438536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65dd0d-afb0-437c-b694-dc657fbb9e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:16.439232 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65dd0d-afb0-437c-b694-dc657fbb9e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:16.440019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41360e-6736-433a-ad16-94cdef928b49 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:16.440973 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65dd0d-afb0-437c-b694-dc657fbb9e59 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:16.448188 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:16.459243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1088/2153] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:13:16 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:16.459243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65dd0d-afb0-437c-b694-dc657fbb9e59 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:16.460236 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1066/2154] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:13:16 2022] PUT /identity/v3/projects/a6dd8f336ea44f14bfaf1c127ddbacb1/users/4f1e5fb6488d4478b8f4449295613e50/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:16.460755 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41360e-6736-433a-ad16-94cdef928b49 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:16.469539 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65dd0d-afb0-437c-b694-dc657fbb9e59 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:16.534763 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705a450-95c8-442e-bf2b-c728988b3fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:16.536148 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705a450-95c8-442e-bf2b-c728988b3fb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:16.536691 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705a450-95c8-442e-bf2b-c728988b3fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:16.537185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705a450-95c8-442e-bf2b-c728988b3fb3 admin admin] PATH_INFO: `/v3/projects/6ec47397315e48549aefc477fee9cc8d/users/74e657f1612a412cb38b50a3c0db24c0/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:16.537679 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41360e-6736-433a-ad16-94cdef928b49 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:16.543156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e99286-a07a-4cc9-944c-8ba324cc719d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:16.543773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e99286-a07a-4cc9-944c-8ba324cc719d admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:16.544055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e99286-a07a-4cc9-944c-8ba324cc719d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:16.544330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e99286-a07a-4cc9-944c-8ba324cc719d admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:16.545018 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e99286-a07a-4cc9-944c-8ba324cc719d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:16.546922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e99286-a07a-4cc9-944c-8ba324cc719d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:16.557690 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705a450-95c8-442e-bf2b-c728988b3fb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ec47397315e48549aefc477fee9cc8d, user_id=74e657f1612a412cb38b50a3c0db24c0, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:16.558587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1067/2155] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:13:16 2022] GET /identity/v3/roles => generated 1427 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:13:16.559868 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02196c2-b6eb-4482-b8b5-a3ea995f660c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:16.560149 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02196c2-b6eb-4482-b8b5-a3ea995f660c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:16.560420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02196c2-b6eb-4482-b8b5-a3ea995f660c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:16.565630 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705a450-95c8-442e-bf2b-c728988b3fb3 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:16.581799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f02196c2-b6eb-4482-b8b5-a3ea995f660c None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:16.582522 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1089/2156] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:13:16 2022] PUT /identity/v3/projects/6ec47397315e48549aefc477fee9cc8d/users/74e657f1612a412cb38b50a3c0db24c0/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:16.582983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f02196c2-b6eb-4482-b8b5-a3ea995f660c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:16.590065 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705a450-95c8-442e-bf2b-c728988b3fb3 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:16.632478 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:16.655722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1068/2157] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:16.660741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee205e-921b-41bb-86ba-0dd92d92bd30 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:16.660741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee205e-921b-41bb-86ba-0dd92d92bd30 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:16.660741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee205e-921b-41bb-86ba-0dd92d92bd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:16.666923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705a450-95c8-442e-bf2b-c728988b3fb3 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:16.671994 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5a399-a153-4506-b824-71c1e9a2c289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:16.672692 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5a399-a153-4506-b824-71c1e9a2c289 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:16.672974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5a399-a153-4506-b824-71c1e9a2c289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:16.673295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5a399-a153-4506-b824-71c1e9a2c289 admin admin] PATH_INFO: `/v3/projects/7a4a7e29ab914a1eb7c37d648b06d980/users/ac253655543a4d639971cf3393f1a585/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:16.689492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d0ee205e-921b-41bb-86ba-0dd92d92bd30 None None] MFA Rules not processed for user `74e657f1612a412cb38b50a3c0db24c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:16.690695 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5a399-a153-4506-b824-71c1e9a2c289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a4a7e29ab914a1eb7c37d648b06d980, user_id=ac253655543a4d639971cf3393f1a585, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:16.690899 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d0ee205e-921b-41bb-86ba-0dd92d92bd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:16.693438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5a399-a153-4506-b824-71c1e9a2c289 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:16.710988 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1090/2158] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:13:16 2022] PUT /identity/v3/projects/7a4a7e29ab914a1eb7c37d648b06d980/users/ac253655543a4d639971cf3393f1a585/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:16.711592 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5a399-a153-4506-b824-71c1e9a2c289 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:16.723288 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-9be5a399-a153-4506-b824-71c1e9a2c289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:16.769818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1069/2159] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:16.774507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4d2cd-5a9f-44bf-9f84-f4fadb66e03d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:16.774804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4d2cd-5a9f-44bf-9f84-f4fadb66e03d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:16.775088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4d2cd-5a9f-44bf-9f84-f4fadb66e03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:16.792872 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5a399-a153-4506-b824-71c1e9a2c289 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:16.800345 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd16561-13d2-424a-9ed2-907f301fe10d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:16.801462 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd16561-13d2-424a-9ed2-907f301fe10d service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:16.801954 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd16561-13d2-424a-9ed2-907f301fe10d service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:16.802449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd16561-13d2-424a-9ed2-907f301fe10d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:16.810867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-dfe4d2cd-5a9f-44bf-9f84-f4fadb66e03d None None] MFA Rules not processed for user `ac253655543a4d639971cf3393f1a585`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:16.813011 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-dfe4d2cd-5a9f-44bf-9f84-f4fadb66e03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:16.831400 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:16.889285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd16561-13d2-424a-9ed2-907f301fe10d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:16.891553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd16561-13d2-424a-9ed2-907f301fe10d service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:16.903495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1091/2160] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:13:16 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:16.906673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd16561-13d2-424a-9ed2-907f301fe10d service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:16.906673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1bd16561-13d2-424a-9ed2-907f301fe10d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:16.913757 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1070/2161] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:16.934949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4d2cd-5a9f-44bf-9f84-f4fadb66e03d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:16.938488 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-dfe4d2cd-5a9f-44bf-9f84-f4fadb66e03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:17.003488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd16561-13d2-424a-9ed2-907f301fe10d service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:17.008013 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d6780-05b9-40f6-8bde-d64ea3f669e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:17.008866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d6780-05b9-40f6-8bde-d64ea3f669e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:17.009266 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d6780-05b9-40f6-8bde-d64ea3f669e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:17.009618 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d6780-05b9-40f6-8bde-d64ea3f669e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:17.028038 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4d2cd-5a9f-44bf-9f84-f4fadb66e03d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:17.032936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328db2e-bbf6-40d4-a868-fc5bfecd3cd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:17.033715 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328db2e-bbf6-40d4-a868-fc5bfecd3cd1 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:17.034075 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328db2e-bbf6-40d4-a868-fc5bfecd3cd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:17.034486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328db2e-bbf6-40d4-a868-fc5bfecd3cd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:17.054246 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328db2e-bbf6-40d4-a868-fc5bfecd3cd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:17.055892 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328db2e-bbf6-40d4-a868-fc5bfecd3cd1 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:17.067680 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1071/2162] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:13:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:13:17.093919 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d6780-05b9-40f6-8bde-d64ea3f669e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:17.095607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d6780-05b9-40f6-8bde-d64ea3f669e4 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:17.098745 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328db2e-bbf6-40d4-a868-fc5bfecd3cd1 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:17.106648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1092/2163] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:13:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:17.112596 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328db2e-bbf6-40d4-a868-fc5bfecd3cd1 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:17.118783 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0017ec2-84d7-4fde-818f-32924c85da14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:17.119449 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0017ec2-84d7-4fde-818f-32924c85da14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:17.119743 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0017ec2-84d7-4fde-818f-32924c85da14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:17.120009 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0017ec2-84d7-4fde-818f-32924c85da14 admin admin] PATH_INFO: `/v3/users/ac253655543a4d639971cf3393f1a585` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:17.121256 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0017ec2-84d7-4fde-818f-32924c85da14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac253655543a4d639971cf3393f1a585)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:17.123224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0017ec2-84d7-4fde-818f-32924c85da14 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:17.132267 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:17.209611 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b0017ec2-84d7-4fde-818f-32924c85da14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac253655543a4d639971cf3393f1a585'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:17.221388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b0017ec2-84d7-4fde-818f-32924c85da14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac253655543a4d639971cf3393f1a585'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:17.222287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b0017ec2-84d7-4fde-818f-32924c85da14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac253655543a4d639971cf3393f1a585'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:17.234850 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b0017ec2-84d7-4fde-818f-32924c85da14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac253655543a4d639971cf3393f1a585'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:17.235576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b0017ec2-84d7-4fde-818f-32924c85da14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac253655543a4d639971cf3393f1a585'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:17.258069 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1072/2164] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:13:17 2022] DELETE /identity/v3/users/ac253655543a4d639971cf3393f1a585 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:13:17.262919 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f5bc2-3508-482c-a9b8-db8e0daf519b None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:17.263223 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f5bc2-3508-482c-a9b8-db8e0daf519b None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:17.263529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f5bc2-3508-482c-a9b8-db8e0daf519b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:17.288716 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-ed5f5bc2-3508-482c-a9b8-db8e0daf519b None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:17.290125 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ed5f5bc2-3508-482c-a9b8-db8e0daf519b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:17.362146 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1093/2165] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:17.374608 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0017ec2-84d7-4fde-818f-32924c85da14 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:17.377324 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b0017ec2-84d7-4fde-818f-32924c85da14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:17.486232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0017ec2-84d7-4fde-818f-32924c85da14 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:17.502095 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6899dc8-aa2f-4690-878a-9c46e84bb405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:17.503367 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6899dc8-aa2f-4690-878a-9c46e84bb405 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:17.504048 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6899dc8-aa2f-4690-878a-9c46e84bb405 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:17.504353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6899dc8-aa2f-4690-878a-9c46e84bb405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:17.517015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6899dc8-aa2f-4690-878a-9c46e84bb405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:17.518508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6899dc8-aa2f-4690-878a-9c46e84bb405 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:17.531783 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1073/2166] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:13:17 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:17.952795 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f5bc2-3508-482c-a9b8-db8e0daf519b None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:17.954649 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ed5f5bc2-3508-482c-a9b8-db8e0daf519b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:17.974758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f5bc2-3508-482c-a9b8-db8e0daf519b None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:17.980385 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326a26c-9251-4490-bd38-9e49ee4fd2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:17.981659 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326a26c-9251-4490-bd38-9e49ee4fd2ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:17.983554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326a26c-9251-4490-bd38-9e49ee4fd2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:17.983554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326a26c-9251-4490-bd38-9e49ee4fd2ae admin admin] PATH_INFO: `/v3/projects/7a4a7e29ab914a1eb7c37d648b06d980` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:17.984303 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d326a26c-9251-4490-bd38-9e49ee4fd2ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a4a7e29ab914a1eb7c37d648b06d980)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:17.986497 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d326a26c-9251-4490-bd38-9e49ee4fd2ae admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:18.003456 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:18.062130 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d326a26c-9251-4490-bd38-9e49ee4fd2ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a4a7e29ab914a1eb7c37d648b06d980', 'request_id': 'req-d326a26c-9251-4490-bd38-9e49ee4fd2ae'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:18.070440 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d326a26c-9251-4490-bd38-9e49ee4fd2ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a4a7e29ab914a1eb7c37d648b06d980', 'request_id': 'req-d326a26c-9251-4490-bd38-9e49ee4fd2ae'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:18.086419 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d326a26c-9251-4490-bd38-9e49ee4fd2ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a4a7e29ab914a1eb7c37d648b06d980', 'request_id': 'req-d326a26c-9251-4490-bd38-9e49ee4fd2ae'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:18.101167 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d326a26c-9251-4490-bd38-9e49ee4fd2ae admin admin] The token cache is being invalidate because project 7a4a7e29ab914a1eb7c37d648b06d980 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:13:18.101593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d326a26c-9251-4490-bd38-9e49ee4fd2ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:18.102355 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d326a26c-9251-4490-bd38-9e49ee4fd2ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:18.105178 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1094/2167] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:13:17 2022] DELETE /identity/v3/projects/7a4a7e29ab914a1eb7c37d648b06d980 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:13:18.110016 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e111f-9cd6-41a8-b777-6345db48b231 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:18.111411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e111f-9cd6-41a8-b777-6345db48b231 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:18.111977 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e111f-9cd6-41a8-b777-6345db48b231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:18.141554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-590e111f-9cd6-41a8-b777-6345db48b231 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:18.142828 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-590e111f-9cd6-41a8-b777-6345db48b231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:18.223151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1074/2168] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:13:18.230951 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69bb9b-9a5a-475a-9f96-703e674aea7f None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:18.231308 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69bb9b-9a5a-475a-9f96-703e674aea7f None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:18.231695 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69bb9b-9a5a-475a-9f96-703e674aea7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:18.260047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-4f69bb9b-9a5a-475a-9f96-703e674aea7f None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:18.261373 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-4f69bb9b-9a5a-475a-9f96-703e674aea7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:18.271086 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1095/2169] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:18.275818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e111f-9cd6-41a8-b777-6345db48b231 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:18.287492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e111f-9cd6-41a8-b777-6345db48b231 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:18.292572 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed62a82-aa2c-4127-96d0-0f418077825e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:18.293596 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed62a82-aa2c-4127-96d0-0f418077825e admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:18.293596 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed62a82-aa2c-4127-96d0-0f418077825e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:18.293828 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed62a82-aa2c-4127-96d0-0f418077825e admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:18.294636 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed62a82-aa2c-4127-96d0-0f418077825e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:13:18.294983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed62a82-aa2c-4127-96d0-0f418077825e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:18.297468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed62a82-aa2c-4127-96d0-0f418077825e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:18.312620 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1075/2170] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:13:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:13:18.318891 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69bb9b-9a5a-475a-9f96-703e674aea7f None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:18.328592 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69bb9b-9a5a-475a-9f96-703e674aea7f None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:18.347923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6fad2e-65b3-462b-953a-51d2381bd609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:18.352117 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fad2e-65b3-462b-953a-51d2381bd609 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:18.352117 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fad2e-65b3-462b-953a-51d2381bd609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:18.352117 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fad2e-65b3-462b-953a-51d2381bd609 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:18.352117 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6fad2e-65b3-462b-953a-51d2381bd609 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:18.352117 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6fad2e-65b3-462b-953a-51d2381bd609 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:18.388358 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1096/2171] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:13:18 2022] POST /identity/v3/projects => generated 380 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:13:18.392328 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed62a82-aa2c-4127-96d0-0f418077825e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:18.403033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed62a82-aa2c-4127-96d0-0f418077825e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:18.415777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:18.479687 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad016a-bef2-4f46-b7b7-732f80d32fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:18.480355 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad016a-bef2-4f46-b7b7-732f80d32fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:18.480700 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad016a-bef2-4f46-b7b7-732f80d32fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:18.480896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad016a-bef2-4f46-b7b7-732f80d32fd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:18.481788 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad016a-bef2-4f46-b7b7-732f80d32fd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:18.484826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad016a-bef2-4f46-b7b7-732f80d32fd3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:18.520272 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1076/2172] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:13:18 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:18.521841 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6fad2e-65b3-462b-953a-51d2381bd609 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:18.537125 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6fad2e-65b3-462b-953a-51d2381bd609 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:18.542704 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e9124-58fd-44f9-8d18-6e535783495a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:18.543410 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e9124-58fd-44f9-8d18-6e535783495a admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:18.543756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e9124-58fd-44f9-8d18-6e535783495a admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:18.544098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e9124-58fd-44f9-8d18-6e535783495a admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:18.545011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e9124-58fd-44f9-8d18-6e535783495a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:18.547544 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e9124-58fd-44f9-8d18-6e535783495a admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:18.559944 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1097/2173] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:13:18 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:18.565152 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad016a-bef2-4f46-b7b7-732f80d32fd3 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:18.582863 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad016a-bef2-4f46-b7b7-732f80d32fd3 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:18.590183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084d852-da83-4309-862b-37506323fd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:18.592100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084d852-da83-4309-862b-37506323fd87 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:18.593828 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084d852-da83-4309-862b-37506323fd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:18.594196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084d852-da83-4309-862b-37506323fd87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:18.594869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084d852-da83-4309-862b-37506323fd87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:18.597335 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084d852-da83-4309-862b-37506323fd87 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:18.613800 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1077/2174] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:13:18 2022] GET /identity/v3/roles => generated 1427 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:18.619568 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e9124-58fd-44f9-8d18-6e535783495a admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:18.643715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e9124-58fd-44f9-8d18-6e535783495a admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:18.654707 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6be8b-c378-4c0e-ae58-d149e997ce1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:18.657564 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6be8b-c378-4c0e-ae58-d149e997ce1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:18.658072 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6be8b-c378-4c0e-ae58-d149e997ce1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:18.659026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6be8b-c378-4c0e-ae58-d149e997ce1d admin admin] PATH_INFO: `/v3/projects/9a4b022cdf9c448dbb51c5ffd9212181/users/d22a27db106e45e68444fa022d254c83/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:18.687591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6be8b-c378-4c0e-ae58-d149e997ce1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a4b022cdf9c448dbb51c5ffd9212181, user_id=d22a27db106e45e68444fa022d254c83, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:18.691311 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6be8b-c378-4c0e-ae58-d149e997ce1d admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:18.710930 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1098/2175] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:13:18 2022] PUT /identity/v3/projects/9a4b022cdf9c448dbb51c5ffd9212181/users/d22a27db106e45e68444fa022d254c83/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:18.716240 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a6305-9d01-4456-8108-206858015eb3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:18.719927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a6305-9d01-4456-8108-206858015eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:18.720217 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a6305-9d01-4456-8108-206858015eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:18.770454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ed8a6305-9d01-4456-8108-206858015eb3 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:18.773561 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ed8a6305-9d01-4456-8108-206858015eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:18.877569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1078/2176] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:18.888983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6be8b-c378-4c0e-ae58-d149e997ce1d admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:18.891500 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f6f6be8b-c378-4c0e-ae58-d149e997ce1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:19.003633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6be8b-c378-4c0e-ae58-d149e997ce1d admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:19.008524 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea36a0-d02d-42d9-b1e4-9a762aad5dc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:19.009301 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea36a0-d02d-42d9-b1e4-9a762aad5dc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:19.009457 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea36a0-d02d-42d9-b1e4-9a762aad5dc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:19.009785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea36a0-d02d-42d9-b1e4-9a762aad5dc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:19.024150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea36a0-d02d-42d9-b1e4-9a762aad5dc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:19.026140 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea36a0-d02d-42d9-b1e4-9a762aad5dc4 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:19.046453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1099/2177] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:13:18 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:22.835808 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6590965-b931-49b5-b9fc-44930d2c3c3a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:22.835808 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6590965-b931-49b5-b9fc-44930d2c3c3a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:22.835808 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6590965-b931-49b5-b9fc-44930d2c3c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:22.926397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e6590965-b931-49b5-b9fc-44930d2c3c3a None None] MFA Rules not processed for user `4f1e5fb6488d4478b8f4449295613e50`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:22.927627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e6590965-b931-49b5-b9fc-44930d2c3c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:23.103859 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1079/2178] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:23.110503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea36a0-d02d-42d9-b1e4-9a762aad5dc4 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:23.113255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-7eea36a0-d02d-42d9-b1e4-9a762aad5dc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:23.131870 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea36a0-d02d-42d9-b1e4-9a762aad5dc4 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:23.173864 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898151f-a3c6-405f-abb6-57a6087f4735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:23.175608 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898151f-a3c6-405f-abb6-57a6087f4735 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:23.178518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898151f-a3c6-405f-abb6-57a6087f4735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:23.178966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898151f-a3c6-405f-abb6-57a6087f4735 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:23.180075 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898151f-a3c6-405f-abb6-57a6087f4735 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:23.183732 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898151f-a3c6-405f-abb6-57a6087f4735 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:23.223312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1100/2179] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:13:23 2022] POST /identity/v3/projects => generated 372 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:23.227272 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6590965-b931-49b5-b9fc-44930d2c3c3a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:23.250948 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6590965-b931-49b5-b9fc-44930d2c3c3a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:23.337782 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:23.480573 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd74a48-b242-4a01-83e5-929cd02196a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:23.481251 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd74a48-b242-4a01-83e5-929cd02196a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:23.481531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd74a48-b242-4a01-83e5-929cd02196a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:23.481802 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd74a48-b242-4a01-83e5-929cd02196a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:23.482648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd74a48-b242-4a01-83e5-929cd02196a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:23.488295 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd74a48-b242-4a01-83e5-929cd02196a1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:23.559803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1080/2180] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:13:23 2022] POST /identity/v3/users => generated 293 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:23.560691 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898151f-a3c6-405f-abb6-57a6087f4735 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:23.589805 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898151f-a3c6-405f-abb6-57a6087f4735 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:23.605777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd29ed7-037e-47bb-ba21-fa0d0454d069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:23.614550 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd29ed7-037e-47bb-ba21-fa0d0454d069 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:23.615752 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd29ed7-037e-47bb-ba21-fa0d0454d069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:23.616063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd29ed7-037e-47bb-ba21-fa0d0454d069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:23.616836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd29ed7-037e-47bb-ba21-fa0d0454d069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:23.627519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd29ed7-037e-47bb-ba21-fa0d0454d069 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:23.658816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:23.685927 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1101/2181] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:13:23 2022] GET /identity/v3/roles => generated 1427 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:23.690448 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd74a48-b242-4a01-83e5-929cd02196a1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:23.710723 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd74a48-b242-4a01-83e5-929cd02196a1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:23.720467 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bdccd-0a7e-4bec-a108-63efc2bf22db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:23.721200 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bdccd-0a7e-4bec-a108-63efc2bf22db admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:23.721943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bdccd-0a7e-4bec-a108-63efc2bf22db admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:23.721943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bdccd-0a7e-4bec-a108-63efc2bf22db admin admin] PATH_INFO: `/v3/projects/55ec13d0e96d4465a6df7b3387d8a757/users/26185d043e5d43c79a0f9df21fd2dcca/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:23.768537 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39bdccd-0a7e-4bec-a108-63efc2bf22db admin admin] RBAC: Authorizing `identity:create_grant(project_id=55ec13d0e96d4465a6df7b3387d8a757, user_id=26185d043e5d43c79a0f9df21fd2dcca, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:23.776703 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39bdccd-0a7e-4bec-a108-63efc2bf22db admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:23.809689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1081/2182] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:13:23 2022] PUT /identity/v3/projects/55ec13d0e96d4465a6df7b3387d8a757/users/26185d043e5d43c79a0f9df21fd2dcca/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:13:23.815455 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd29ed7-037e-47bb-ba21-fa0d0454d069 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:23.835535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd29ed7-037e-47bb-ba21-fa0d0454d069 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:24.015175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63a6aa-1930-45d2-9bbc-687d56ca89e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:24.016579 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63a6aa-1930-45d2-9bbc-687d56ca89e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:24.017567 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63a6aa-1930-45d2-9bbc-687d56ca89e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:24.017932 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63a6aa-1930-45d2-9bbc-687d56ca89e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:24.019340 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63a6aa-1930-45d2-9bbc-687d56ca89e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:24.021287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63a6aa-1930-45d2-9bbc-687d56ca89e6 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:24.052260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1102/2183] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:13:23 2022] GET /identity/v3/roles => generated 1427 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:24.053551 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bdccd-0a7e-4bec-a108-63efc2bf22db admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:24.069350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bdccd-0a7e-4bec-a108-63efc2bf22db admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:24.077352 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21659f-d68c-418f-a25a-cde9eeedde53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:24.078351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21659f-d68c-418f-a25a-cde9eeedde53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:24.079518 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21659f-d68c-418f-a25a-cde9eeedde53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:24.080905 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21659f-d68c-418f-a25a-cde9eeedde53 admin admin] PATH_INFO: `/v3/projects/55ec13d0e96d4465a6df7b3387d8a757/users/26185d043e5d43c79a0f9df21fd2dcca/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:24.089200 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21659f-d68c-418f-a25a-cde9eeedde53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55ec13d0e96d4465a6df7b3387d8a757, user_id=26185d043e5d43c79a0f9df21fd2dcca, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:24.101958 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21659f-d68c-418f-a25a-cde9eeedde53 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:24.148367 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1082/2184] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:13:24 2022] PUT /identity/v3/projects/55ec13d0e96d4465a6df7b3387d8a757/users/26185d043e5d43c79a0f9df21fd2dcca/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:25.685255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d173801-3a9f-4502-be25-01d30c5ab0df None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:25.686213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d173801-3a9f-4502-be25-01d30c5ab0df None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:25.686650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d173801-3a9f-4502-be25-01d30c5ab0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:25.728597 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-4d173801-3a9f-4502-be25-01d30c5ab0df None None] MFA Rules not processed for user `6c30f2ab48e74d4690aafe46c4bf682f`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:25.734945 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-4d173801-3a9f-4502-be25-01d30c5ab0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:25.853370 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1103/2185] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:25 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:28.627372 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3590c97-cee1-40db-881f-d5a3a648982e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:28.629459 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3590c97-cee1-40db-881f-d5a3a648982e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:28.629983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3590c97-cee1-40db-881f-d5a3a648982e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:28.780261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a3590c97-cee1-40db-881f-d5a3a648982e None None] MFA Rules not processed for user `d22a27db106e45e68444fa022d254c83`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:28.785007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a3590c97-cee1-40db-881f-d5a3a648982e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:28.945822 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1083/2186] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:13:28.950337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d173801-3a9f-4502-be25-01d30c5ab0df None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:29.091282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d173801-3a9f-4502-be25-01d30c5ab0df None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:29.099731 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d082e5e-a623-4539-afb4-1b103d2f2f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:29.100520 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d082e5e-a623-4539-afb4-1b103d2f2f14 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:29.100897 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d082e5e-a623-4539-afb4-1b103d2f2f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:29.101362 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d082e5e-a623-4539-afb4-1b103d2f2f14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:29.102158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d082e5e-a623-4539-afb4-1b103d2f2f14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:29.104349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d082e5e-a623-4539-afb4-1b103d2f2f14 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:29.150582 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1104/2187] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:13:28 2022] POST /identity/v3/projects => generated 380 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:29.159858 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3590c97-cee1-40db-881f-d5a3a648982e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:29.174604 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3590c97-cee1-40db-881f-d5a3a648982e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:29.271320 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b971e-608f-4907-bea7-907c16a9cd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:29.271965 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b971e-608f-4907-bea7-907c16a9cd6e admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:29.272252 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b971e-608f-4907-bea7-907c16a9cd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:29.272528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b971e-608f-4907-bea7-907c16a9cd6e admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:29.273256 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b971e-608f-4907-bea7-907c16a9cd6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:29.275227 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b971e-608f-4907-bea7-907c16a9cd6e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:29.290944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:29.311524 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1084/2188] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:13:29 2022] POST /identity/v3/users => generated 297 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:29.315536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d082e5e-a623-4539-afb4-1b103d2f2f14 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:29.331070 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d082e5e-a623-4539-afb4-1b103d2f2f14 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:29.337361 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd327c0-f104-412b-9712-d675a8ecbddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:29.340829 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd327c0-f104-412b-9712-d675a8ecbddb admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:29.341408 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd327c0-f104-412b-9712-d675a8ecbddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:29.342660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd327c0-f104-412b-9712-d675a8ecbddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:29.343648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd327c0-f104-412b-9712-d675a8ecbddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:29.346123 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd327c0-f104-412b-9712-d675a8ecbddb admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:29.370906 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1105/2189] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:13:29 2022] GET /identity/v3/roles => generated 1427 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:29.376188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b971e-608f-4907-bea7-907c16a9cd6e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:29.388423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b971e-608f-4907-bea7-907c16a9cd6e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:29.396053 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29005375-30de-4e54-9493-8dae663f5a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:29.397369 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29005375-30de-4e54-9493-8dae663f5a76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:29.397910 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29005375-30de-4e54-9493-8dae663f5a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:29.398393 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29005375-30de-4e54-9493-8dae663f5a76 admin admin] PATH_INFO: `/v3/projects/29849de6f58e47e7b7ba96234cbfe1b2/users/6e14cfb4e8fc4c2e8a3f3fd5aa8a7cde/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:29.424222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29005375-30de-4e54-9493-8dae663f5a76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29849de6f58e47e7b7ba96234cbfe1b2, user_id=6e14cfb4e8fc4c2e8a3f3fd5aa8a7cde, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:29.427864 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29005375-30de-4e54-9493-8dae663f5a76 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:29.452202 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd327c0-f104-412b-9712-d675a8ecbddb admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:29.452918 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1085/2190] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:13:29 2022] PUT /identity/v3/projects/29849de6f58e47e7b7ba96234cbfe1b2/users/6e14cfb4e8fc4c2e8a3f3fd5aa8a7cde/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:29.465277 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd327c0-f104-412b-9712-d675a8ecbddb admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:29.484300 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:29.578577 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07651777-b370-4539-b24c-8eac0e55e0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:29.579875 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07651777-b370-4539-b24c-8eac0e55e0fd admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:29.580541 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07651777-b370-4539-b24c-8eac0e55e0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:29.581430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07651777-b370-4539-b24c-8eac0e55e0fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:29.585074 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07651777-b370-4539-b24c-8eac0e55e0fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:29.587852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07651777-b370-4539-b24c-8eac0e55e0fd admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:29.633665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29005375-30de-4e54-9493-8dae663f5a76 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:29.635963 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1106/2191] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:13:29 2022] GET /identity/v3/roles => generated 1427 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:29.653036 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29005375-30de-4e54-9493-8dae663f5a76 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:29.665011 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ab724-afbf-4838-a427-8ea8661acc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:29.678551 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ab724-afbf-4838-a427-8ea8661acc69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:29.679497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ab724-afbf-4838-a427-8ea8661acc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:29.683835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ab724-afbf-4838-a427-8ea8661acc69 admin admin] PATH_INFO: `/v3/projects/29849de6f58e47e7b7ba96234cbfe1b2/users/6e14cfb4e8fc4c2e8a3f3fd5aa8a7cde/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:29.688537 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ab724-afbf-4838-a427-8ea8661acc69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29849de6f58e47e7b7ba96234cbfe1b2, user_id=6e14cfb4e8fc4c2e8a3f3fd5aa8a7cde, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:29.698392 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ab724-afbf-4838-a427-8ea8661acc69 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:29.736734 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1086/2192] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:13:29 2022] PUT /identity/v3/projects/29849de6f58e47e7b7ba96234cbfe1b2/users/6e14cfb4e8fc4c2e8a3f3fd5aa8a7cde/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:32.081198 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07651777-b370-4539-b24c-8eac0e55e0fd admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:32.085335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-07651777-b370-4539-b24c-8eac0e55e0fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:32.100267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07651777-b370-4539-b24c-8eac0e55e0fd admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:32.179243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89be4d-1803-49c6-ae44-f798da799f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:32.180298 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89be4d-1803-49c6-ae44-f798da799f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:32.180950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89be4d-1803-49c6-ae44-f798da799f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:32.182195 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89be4d-1803-49c6-ae44-f798da799f4b admin admin] PATH_INFO: `/v3/users/74e657f1612a412cb38b50a3c0db24c0` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:32.184050 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e89be4d-1803-49c6-ae44-f798da799f4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=74e657f1612a412cb38b50a3c0db24c0)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:32.186616 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e89be4d-1803-49c6-ae44-f798da799f4b admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:32.287359 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-4e89be4d-1803-49c6-ae44-f798da799f4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74e657f1612a412cb38b50a3c0db24c0'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:32.288876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-4e89be4d-1803-49c6-ae44-f798da799f4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74e657f1612a412cb38b50a3c0db24c0'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:32.302297 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-4e89be4d-1803-49c6-ae44-f798da799f4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74e657f1612a412cb38b50a3c0db24c0'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:32.324971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-4e89be4d-1803-49c6-ae44-f798da799f4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74e657f1612a412cb38b50a3c0db24c0'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:32.337974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-4e89be4d-1803-49c6-ae44-f798da799f4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74e657f1612a412cb38b50a3c0db24c0'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:32.351775 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1107/2193] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:13:32 2022] DELETE /identity/v3/users/74e657f1612a412cb38b50a3c0db24c0 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:32.354298 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dff2c-bdd0-42fa-b1c7-6233a146077d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:32.355312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dff2c-bdd0-42fa-b1c7-6233a146077d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:32.355832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dff2c-bdd0-42fa-b1c7-6233a146077d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:32.416863 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ec8dff2c-bdd0-42fa-b1c7-6233a146077d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:32.422371 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ec8dff2c-bdd0-42fa-b1c7-6233a146077d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:32.600629 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1087/2194] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:32.620825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89be4d-1803-49c6-ae44-f798da799f4b admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:32.622805 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-4e89be4d-1803-49c6-ae44-f798da799f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:32.718129 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:32.760316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89be4d-1803-49c6-ae44-f798da799f4b admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:32.772192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8367272-217d-4067-bcbf-c752dc5ba9cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:32.772880 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8367272-217d-4067-bcbf-c752dc5ba9cb service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:32.773205 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8367272-217d-4067-bcbf-c752dc5ba9cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:32.774299 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8367272-217d-4067-bcbf-c752dc5ba9cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:32.814548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8367272-217d-4067-bcbf-c752dc5ba9cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:32.817096 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8367272-217d-4067-bcbf-c752dc5ba9cb service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:32.832057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1108/2195] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:13:32 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:34.073979 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8dff2c-bdd0-42fa-b1c7-6233a146077d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:34.075663 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ec8dff2c-bdd0-42fa-b1c7-6233a146077d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:34.112619 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8dff2c-bdd0-42fa-b1c7-6233a146077d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:34.136751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf44317-2590-46ce-9df8-e003a0fd00a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:34.138711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf44317-2590-46ce-9df8-e003a0fd00a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:34.139366 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf44317-2590-46ce-9df8-e003a0fd00a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:34.139798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf44317-2590-46ce-9df8-e003a0fd00a6 admin admin] PATH_INFO: `/v3/projects/6ec47397315e48549aefc477fee9cc8d` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:34.141468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf44317-2590-46ce-9df8-e003a0fd00a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ec47397315e48549aefc477fee9cc8d)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:34.143638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf44317-2590-46ce-9df8-e003a0fd00a6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:34.262007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ebf44317-2590-46ce-9df8-e003a0fd00a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ec47397315e48549aefc477fee9cc8d', 'request_id': 'req-ebf44317-2590-46ce-9df8-e003a0fd00a6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:34.274238 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ebf44317-2590-46ce-9df8-e003a0fd00a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ec47397315e48549aefc477fee9cc8d', 'request_id': 'req-ebf44317-2590-46ce-9df8-e003a0fd00a6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:34.299160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ebf44317-2590-46ce-9df8-e003a0fd00a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ec47397315e48549aefc477fee9cc8d', 'request_id': 'req-ebf44317-2590-46ce-9df8-e003a0fd00a6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:34.336303 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ebf44317-2590-46ce-9df8-e003a0fd00a6 admin admin] The token cache is being invalidate because project 6ec47397315e48549aefc477fee9cc8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:13:34.336806 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ebf44317-2590-46ce-9df8-e003a0fd00a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:34.338505 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ebf44317-2590-46ce-9df8-e003a0fd00a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:34.346859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bfcc1-fb22-4e69-bb7b-65ab5bf9a8ad None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:34.347367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bfcc1-fb22-4e69-bb7b-65ab5bf9a8ad None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:34.347449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bfcc1-fb22-4e69-bb7b-65ab5bf9a8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:34.355317 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1088/2196] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:13:34 2022] DELETE /identity/v3/projects/6ec47397315e48549aefc477fee9cc8d => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:34.408321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-8f3bfcc1-fb22-4e69-bb7b-65ab5bf9a8ad None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:34.423903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-8f3bfcc1-fb22-4e69-bb7b-65ab5bf9a8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:34.571847 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1109/2197] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:34.583766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36658573-a6c9-4343-89c2-551a1922aeef None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:34.584066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36658573-a6c9-4343-89c2-551a1922aeef None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:34.584349 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36658573-a6c9-4343-89c2-551a1922aeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:34.623702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-36658573-a6c9-4343-89c2-551a1922aeef None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:34.626533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-36658573-a6c9-4343-89c2-551a1922aeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:34.671189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1089/2198] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:34.674505 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3bfcc1-fb22-4e69-bb7b-65ab5bf9a8ad None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:34.708832 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3bfcc1-fb22-4e69-bb7b-65ab5bf9a8ad None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:34.722416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd8de5-9866-4b3a-a147-adc5a74e85ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:34.724008 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd8de5-9866-4b3a-a147-adc5a74e85ac admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:34.724510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd8de5-9866-4b3a-a147-adc5a74e85ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:34.724999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd8de5-9866-4b3a-a147-adc5a74e85ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:34.726351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd8de5-9866-4b3a-a147-adc5a74e85ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:13:34.730423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd8de5-9866-4b3a-a147-adc5a74e85ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:34.735010 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd8de5-9866-4b3a-a147-adc5a74e85ac admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:34.797259 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1110/2199] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:13:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:13:34.800430 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36658573-a6c9-4343-89c2-551a1922aeef None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:34.819650 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36658573-a6c9-4343-89c2-551a1922aeef None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:34.835329 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b8426-a63c-413b-9cfd-136fc80ed7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:34.835971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b8426-a63c-413b-9cfd-136fc80ed7eb admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:34.836249 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b8426-a63c-413b-9cfd-136fc80ed7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:34.836714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b8426-a63c-413b-9cfd-136fc80ed7eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:34.837568 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b8426-a63c-413b-9cfd-136fc80ed7eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:34.839513 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b8426-a63c-413b-9cfd-136fc80ed7eb admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:34.953299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1090/2200] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:13:34 2022] POST /identity/v3/projects => generated 384 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:34.957346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd8de5-9866-4b3a-a147-adc5a74e85ac admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:34.981580 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd8de5-9866-4b3a-a147-adc5a74e85ac admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:35.066350 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a35db-e020-454b-a125-ad5647809f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:35.067135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a35db-e020-454b-a125-ad5647809f07 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:35.067619 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a35db-e020-454b-a125-ad5647809f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:35.068506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a35db-e020-454b-a125-ad5647809f07 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:35.069446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154a35db-e020-454b-a125-ad5647809f07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:35.071561 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154a35db-e020-454b-a125-ad5647809f07 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:35.110987 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1111/2201] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:13:34 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:35.114117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b8426-a63c-413b-9cfd-136fc80ed7eb admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:35.128239 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b8426-a63c-413b-9cfd-136fc80ed7eb admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:35.135403 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f7294-8bff-453c-86b8-aa8f69d6a616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:35.136226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f7294-8bff-453c-86b8-aa8f69d6a616 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:35.136847 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f7294-8bff-453c-86b8-aa8f69d6a616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:35.137291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f7294-8bff-453c-86b8-aa8f69d6a616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:35.138126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f7294-8bff-453c-86b8-aa8f69d6a616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:35.140876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f7294-8bff-453c-86b8-aa8f69d6a616 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:35.149958 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:35.165432 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1091/2202] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:13:35 2022] GET /identity/v3/roles => generated 1427 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:35.167023 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a35db-e020-454b-a125-ad5647809f07 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:35.189092 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a35db-e020-454b-a125-ad5647809f07 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:35.195182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31297978-3b8d-4883-a40d-aa43e137d28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:35.196067 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31297978-3b8d-4883-a40d-aa43e137d28e admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:35.196460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31297978-3b8d-4883-a40d-aa43e137d28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:35.196879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31297978-3b8d-4883-a40d-aa43e137d28e admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:35.197733 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31297978-3b8d-4883-a40d-aa43e137d28e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:35.199834 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31297978-3b8d-4883-a40d-aa43e137d28e admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:35.216939 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1112/2203] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:13:35 2022] GET /identity/v3/roles => generated 1427 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:35.218402 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f7294-8bff-453c-86b8-aa8f69d6a616 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:35.233940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f7294-8bff-453c-86b8-aa8f69d6a616 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:35.240082 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83427a-819b-44a5-9844-c45fe612cf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:35.240979 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83427a-819b-44a5-9844-c45fe612cf3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:35.242685 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83427a-819b-44a5-9844-c45fe612cf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:35.243093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83427a-819b-44a5-9844-c45fe612cf3b admin admin] PATH_INFO: `/v3/projects/0d0ba64c011a419b9cc8f5501a8606ac/users/b074a126a458485e882a9e9eea178dca/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:35.267762 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83427a-819b-44a5-9844-c45fe612cf3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d0ba64c011a419b9cc8f5501a8606ac, user_id=b074a126a458485e882a9e9eea178dca, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:35.271539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83427a-819b-44a5-9844-c45fe612cf3b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:35.298465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1092/2204] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:13:35 2022] PUT /identity/v3/projects/0d0ba64c011a419b9cc8f5501a8606ac/users/b074a126a458485e882a9e9eea178dca/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:13:35.301011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badeb7f-4a0a-4c3f-823c-6a3cae6f5cc2 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:35.301688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badeb7f-4a0a-4c3f-823c-6a3cae6f5cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:35.302251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badeb7f-4a0a-4c3f-823c-6a3cae6f5cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:35.344171 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-6badeb7f-4a0a-4c3f-823c-6a3cae6f5cc2 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:35.345824 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-6badeb7f-4a0a-4c3f-823c-6a3cae6f5cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:35.393337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:35.450208 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1113/2205] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:35.461655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83427a-819b-44a5-9844-c45fe612cf3b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:35.464079 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6e83427a-819b-44a5-9844-c45fe612cf3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:35.571381 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83427a-819b-44a5-9844-c45fe612cf3b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:35.579200 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31d380-33d3-4284-b47e-b5db5154d0ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:35.580747 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31d380-33d3-4284-b47e-b5db5154d0ff service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:35.581290 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31d380-33d3-4284-b47e-b5db5154d0ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:35.581776 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31d380-33d3-4284-b47e-b5db5154d0ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:35.599225 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31d380-33d3-4284-b47e-b5db5154d0ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:35.602190 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31d380-33d3-4284-b47e-b5db5154d0ff service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:35.617507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1093/2206] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:13:35 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:37.197206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edb36f-c31d-443f-9c82-3d789ecb5be4 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:37.197729 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edb36f-c31d-443f-9c82-3d789ecb5be4 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:37.199073 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edb36f-c31d-443f-9c82-3d789ecb5be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:37.243479 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-83edb36f-c31d-443f-9c82-3d789ecb5be4 None None] MFA Rules not processed for user `b074a126a458485e882a9e9eea178dca`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:37.245820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-83edb36f-c31d-443f-9c82-3d789ecb5be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:37.377194 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1114/2207] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:37.386413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31d380-33d3-4284-b47e-b5db5154d0ff service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:37.389747 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-eb31d380-33d3-4284-b47e-b5db5154d0ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:37.489034 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31d380-33d3-4284-b47e-b5db5154d0ff service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:37.493487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161572d-a966-49bd-abec-d977618827e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:37.494128 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161572d-a966-49bd-abec-d977618827e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:37.494415 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161572d-a966-49bd-abec-d977618827e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:37.495006 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161572d-a966-49bd-abec-d977618827e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:37.506096 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8161572d-a966-49bd-abec-d977618827e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:37.509659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8161572d-a966-49bd-abec-d977618827e2 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:37.525267 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1094/2208] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:13:37 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:39.328723 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430b427-2e22-451b-a50d-f8d3e548fa17 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:39.329270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430b427-2e22-451b-a50d-f8d3e548fa17 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:39.329708 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430b427-2e22-451b-a50d-f8d3e548fa17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:39.382209 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-7430b427-2e22-451b-a50d-f8d3e548fa17 None None] MFA Rules not processed for user `b074a126a458485e882a9e9eea178dca`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:39.385402 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-7430b427-2e22-451b-a50d-f8d3e548fa17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:39.399681 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1115/2209] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:39.860295 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed80e86-af12-4e89-be6a-4f8bf6aaf4ff None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:39.861214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed80e86-af12-4e89-be6a-4f8bf6aaf4ff None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:39.861654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed80e86-af12-4e89-be6a-4f8bf6aaf4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:39.910305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5ed80e86-af12-4e89-be6a-4f8bf6aaf4ff None None] MFA Rules not processed for user `c9633b9cbd8a4332a605ca66d73dc5b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:39.912552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5ed80e86-af12-4e89-be6a-4f8bf6aaf4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:40.100569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1095/2210] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:41.665575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab6115-73ef-4097-a886-3eb3c5902016 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:41.667032 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab6115-73ef-4097-a886-3eb3c5902016 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:41.667485 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab6115-73ef-4097-a886-3eb3c5902016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:41.728189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-a1ab6115-73ef-4097-a886-3eb3c5902016 None None] MFA Rules not processed for user `26185d043e5d43c79a0f9df21fd2dcca`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:41.731163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-a1ab6115-73ef-4097-a886-3eb3c5902016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:41.957523 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1116/2211] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:42.020432 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed80e86-af12-4e89-be6a-4f8bf6aaf4ff None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:42.023014 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5ed80e86-af12-4e89-be6a-4f8bf6aaf4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:42.084449 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:42.166100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed80e86-af12-4e89-be6a-4f8bf6aaf4ff None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:42.173509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8d150-bf25-4121-88e5-61f3940a465b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:42.174158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8d150-bf25-4121-88e5-61f3940a465b service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:42.174442 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8d150-bf25-4121-88e5-61f3940a465b service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:42.174718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8d150-bf25-4121-88e5-61f3940a465b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:42.176643 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-06d8d150-bf25-4121-88e5-61f3940a465b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:42.192541 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ed7af-5415-4dec-bdf8-631e28d6f611 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:42.192924 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ed7af-5415-4dec-bdf8-631e28d6f611 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:42.193313 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ed7af-5415-4dec-bdf8-631e28d6f611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:42.249173 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-a09ed7af-5415-4dec-bdf8-631e28d6f611 None None] MFA Rules not processed for user `6e14cfb4e8fc4c2e8a3f3fd5aa8a7cde`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:42.250562 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-a09ed7af-5415-4dec-bdf8-631e28d6f611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:42.309583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:42.317105 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8d150-bf25-4121-88e5-61f3940a465b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:42.321496 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8d150-bf25-4121-88e5-61f3940a465b service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:42.352092 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1096/2212] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:13:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:42.369709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1117/2213] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:42 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:13:42.390311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8d150-bf25-4121-88e5-61f3940a465b service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:42.391818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-06d8d150-bf25-4121-88e5-61f3940a465b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:42.423243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8d150-bf25-4121-88e5-61f3940a465b service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:42.432277 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a51f44-6132-4598-864e-b7f24e3b9c15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:42.432923 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a51f44-6132-4598-864e-b7f24e3b9c15 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:42.433254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a51f44-6132-4598-864e-b7f24e3b9c15 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:42.433535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a51f44-6132-4598-864e-b7f24e3b9c15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:42.435655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d3a51f44-6132-4598-864e-b7f24e3b9c15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:42.610983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a51f44-6132-4598-864e-b7f24e3b9c15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:42.613357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a51f44-6132-4598-864e-b7f24e3b9c15 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:42.652531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1097/2214] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:13:42 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:46.067537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200256d-b804-4434-84df-58da8733e7cd None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:46.068118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200256d-b804-4434-84df-58da8733e7cd None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:46.068231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200256d-b804-4434-84df-58da8733e7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:46.108926 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-1200256d-b804-4434-84df-58da8733e7cd None None] MFA Rules not processed for user `4f1e5fb6488d4478b8f4449295613e50`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:46.109971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1200256d-b804-4434-84df-58da8733e7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:46.130418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1118/2215] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:13:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:52.630900 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=72 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:56.091875 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=15 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:14:06.168435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302d584-2331-4471-beec-e7b301a7da45 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:06.169017 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302d584-2331-4471-beec-e7b301a7da45 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:06.169017 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302d584-2331-4471-beec-e7b301a7da45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:06.196510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b302d584-2331-4471-beec-e7b301a7da45 None None] MFA Rules not processed for user `1cd36b543bd9487ebd8ce578e9a76c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:14:06.198776 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b302d584-2331-4471-beec-e7b301a7da45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:06.296883 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1098/2216] 10.222.0.100 () {58 vars in 1014 bytes} [Tue Aug 2 09:14:06 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:14:06.369869 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200256d-b804-4434-84df-58da8733e7cd None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:06.371303 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1200256d-b804-4434-84df-58da8733e7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:06.476312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200256d-b804-4434-84df-58da8733e7cd None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:06.483397 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266f891-2d9c-4e97-9ea0-aa9f03d52959 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50cb532b58ed4951ba960a0a7d579b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:06.484351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266f891-2d9c-4e97-9ea0-aa9f03d52959 service swift] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:06.484798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266f891-2d9c-4e97-9ea0-aa9f03d52959 service swift] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:06.485266 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266f891-2d9c-4e97-9ea0-aa9f03d52959 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:06.498415 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266f891-2d9c-4e97-9ea0-aa9f03d52959 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:06.502032 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266f891-2d9c-4e97-9ea0-aa9f03d52959 service swift] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:06.525069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1119/2217] 10.222.0.100 () {60 vars in 1389 bytes} [Tue Aug 2 09:14:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:14:16.275264 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=25 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:14:16.510612 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=32 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:14:20.359018 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b302d584-2331-4471-beec-e7b301a7da45 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:20.360504 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b302d584-2331-4471-beec-e7b301a7da45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:20.441956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b302d584-2331-4471-beec-e7b301a7da45 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:20.447217 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696c679-9d19-4fb9-9a04-717bfa9f900e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b40981fe179a43dc81553d3594dad649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:20.447857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696c679-9d19-4fb9-9a04-717bfa9f900e service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:20.448145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696c679-9d19-4fb9-9a04-717bfa9f900e service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:20.448427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696c679-9d19-4fb9-9a04-717bfa9f900e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:20.451138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9696c679-9d19-4fb9-9a04-717bfa9f900e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:20.463319 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696c679-9d19-4fb9-9a04-717bfa9f900e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:20.465251 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696c679-9d19-4fb9-9a04-717bfa9f900e service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:20.475838 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1099/2218] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:14:20 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:14:22.039917 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4aa00-c3d2-44a6-a069-a733dcd38ab9 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:22.040524 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4aa00-c3d2-44a6-a069-a733dcd38ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:22.040634 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4aa00-c3d2-44a6-a069-a733dcd38ab9 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:22.041953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1120/2219] 10.222.0.100 () {54 vars in 926 bytes} [Tue Aug 2 09:14:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:14:22.049531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8867bd6-785a-4e3c-9c1d-5dae1cc93bed None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:22.050094 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8867bd6-785a-4e3c-9c1d-5dae1cc93bed None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:22.050094 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8867bd6-785a-4e3c-9c1d-5dae1cc93bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:22.072766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a8867bd6-785a-4e3c-9c1d-5dae1cc93bed None None] MFA Rules not processed for user `49a0c1a4a6cf43e28ec1b8488c681848`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:14:22.074545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a8867bd6-785a-4e3c-9c1d-5dae1cc93bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:22.091398 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1100/2220] 10.222.0.100 () {58 vars in 1023 bytes} [Tue Aug 2 09:14:22 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:14:22.103221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4aa00-c3d2-44a6-a069-a733dcd38ab9 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:22.105031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-6ee4aa00-c3d2-44a6-a069-a733dcd38ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:22.170896 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4aa00-c3d2-44a6-a069-a733dcd38ab9 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:22.174769 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9722c44-6222-44b4-9ff5-d835ae9e9448 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd959ddd087724547a11c3d4c796d8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:22.175412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9722c44-6222-44b4-9ff5-d835ae9e9448 service glance] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:22.175695 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9722c44-6222-44b4-9ff5-d835ae9e9448 service glance] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:22.175968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9722c44-6222-44b4-9ff5-d835ae9e9448 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:22.201720 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9722c44-6222-44b4-9ff5-d835ae9e9448 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:22.201720 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9722c44-6222-44b4-9ff5-d835ae9e9448 service glance] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:22.208289 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1121/2221] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:14:22 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:14:22.284416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c4ed3-8c40-49f4-a952-be4a9b09a219 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:22.284649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c4ed3-8c40-49f4-a952-be4a9b09a219 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:22.284937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c4ed3-8c40-49f4-a952-be4a9b09a219 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:22.286383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1101/2222] 10.222.0.100 () {54 vars in 926 bytes} [Tue Aug 2 09:14:22 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:14:22.291381 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa46be-9f41-48e0-82a2-0d0c621a51d2 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:22.291674 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa46be-9f41-48e0-82a2-0d0c621a51d2 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:22.291953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa46be-9f41-48e0-82a2-0d0c621a51d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:22.313588 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-0ffa46be-9f41-48e0-82a2-0d0c621a51d2 None None] MFA Rules not processed for user `f4afef33deb247509765a8c88a78ac39`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:14:22.314901 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-0ffa46be-9f41-48e0-82a2-0d0c621a51d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:22.321660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1122/2223] 10.222.0.100 () {58 vars in 1023 bytes} [Tue Aug 2 09:14:22 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:14:22.334343 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c4ed3-8c40-49f4-a952-be4a9b09a219 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:22.336356 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-613c4ed3-8c40-49f4-a952-be4a9b09a219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:22.350393 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c4ed3-8c40-49f4-a952-be4a9b09a219 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:22.354008 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c7a91-2930-4891-a5ab-d15e047341af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:22.354630 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c7a91-2930-4891-a5ab-d15e047341af service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:22.354914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c7a91-2930-4891-a5ab-d15e047341af service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:22.355221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c7a91-2930-4891-a5ab-d15e047341af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:22.363539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c7a91-2930-4891-a5ab-d15e047341af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:22.364971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c7a91-2930-4891-a5ab-d15e047341af service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:22.373552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1102/2224] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:14:22 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:14:22.461012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc690194-658e-4bbd-98c7-09d7294cd5ad None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:22.461499 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc690194-658e-4bbd-98c7-09d7294cd5ad None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:22.461789 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc690194-658e-4bbd-98c7-09d7294cd5ad None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:22.463266 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1123/2225] 10.222.0.100 () {54 vars in 926 bytes} [Tue Aug 2 09:14:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:14:22.469100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252569a2-e23d-412e-b144-9fb5155a9982 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:22.469420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252569a2-e23d-412e-b144-9fb5155a9982 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:22.469703 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252569a2-e23d-412e-b144-9fb5155a9982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:22.490367 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-252569a2-e23d-412e-b144-9fb5155a9982 None None] MFA Rules not processed for user `49a0c1a4a6cf43e28ec1b8488c681848`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:14:22.491665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-252569a2-e23d-412e-b144-9fb5155a9982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:22.503233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1103/2226] 10.222.0.100 () {58 vars in 1023 bytes} [Tue Aug 2 09:14:22 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:14:22.515049 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc690194-658e-4bbd-98c7-09d7294cd5ad None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:22.529574 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc690194-658e-4bbd-98c7-09d7294cd5ad None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:22.533289 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06e729-d211-40fd-a71f-ae1416ae737a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:22.533909 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06e729-d211-40fd-a71f-ae1416ae737a service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:22.534191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06e729-d211-40fd-a71f-ae1416ae737a service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:22.534466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06e729-d211-40fd-a71f-ae1416ae737a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:22.545792 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06e729-d211-40fd-a71f-ae1416ae737a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:22.547234 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06e729-d211-40fd-a71f-ae1416ae737a service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:22.559413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1124/2227] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:14:22 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:14:26.335130 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a724e-b097-4f95-9a46-4127231bdcb0 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:26.335765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a724e-b097-4f95-9a46-4127231bdcb0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:26.335765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a724e-b097-4f95-9a46-4127231bdcb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:26.372426 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d58a724e-b097-4f95-9a46-4127231bdcb0 None None] MFA Rules not processed for user `d22a27db106e45e68444fa022d254c83`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:14:26.374865 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d58a724e-b097-4f95-9a46-4127231bdcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:26.383771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1104/2228] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:14:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:14:27.079128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0fda8-3ddc-4f5e-8a18-f29bce8826ae None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:27.080138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0fda8-3ddc-4f5e-8a18-f29bce8826ae None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:27.080421 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0fda8-3ddc-4f5e-8a18-f29bce8826ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:27.102794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-fba0fda8-3ddc-4f5e-8a18-f29bce8826ae None None] MFA Rules not processed for user `1cd36b543bd9487ebd8ce578e9a76c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:14:27.104177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-fba0fda8-3ddc-4f5e-8a18-f29bce8826ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:27.112686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1125/2229] 10.222.0.100 () {58 vars in 1014 bytes} [Tue Aug 2 09:14:27 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:14:27.126942 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a724e-b097-4f95-9a46-4127231bdcb0 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:27.140735 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a724e-b097-4f95-9a46-4127231bdcb0 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:27.147911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229dcffe-3e6e-485e-b584-09a89b01b4fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50cb532b58ed4951ba960a0a7d579b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:27.148836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dcffe-3e6e-485e-b584-09a89b01b4fa service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:27.149141 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dcffe-3e6e-485e-b584-09a89b01b4fa service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:27.149592 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dcffe-3e6e-485e-b584-09a89b01b4fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:27.160061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229dcffe-3e6e-485e-b584-09a89b01b4fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:27.161872 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229dcffe-3e6e-485e-b584-09a89b01b4fa service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:27.175371 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1105/2230] 10.222.0.100 () {60 vars in 1389 bytes} [Tue Aug 2 09:14:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 02 09:14:37.097748 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=38 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:14:37.168080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=53 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:14:49.263199 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3410b9-37d1-419c-9fb5-a85f0b4a9077 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:49.263799 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3410b9-37d1-419c-9fb5-a85f0b4a9077 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:49.263799 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3410b9-37d1-419c-9fb5-a85f0b4a9077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:49.283580 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-fd3410b9-37d1-419c-9fb5-a85f0b4a9077 None None] MFA Rules not processed for user `1cd36b543bd9487ebd8ce578e9a76c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:14:49.284781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-fd3410b9-37d1-419c-9fb5-a85f0b4a9077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:49.301662 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1126/2231] 10.222.0.100 () {58 vars in 972 bytes} [Tue Aug 2 09:14:49 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:14:49.315366 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229dcffe-3e6e-485e-b584-09a89b01b4fa service swift] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:49.326958 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229dcffe-3e6e-485e-b584-09a89b01b4fa service swift] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:49.332119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13557c-4675-4798-8630-338056b7370a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50cb532b58ed4951ba960a0a7d579b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:49.332746 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13557c-4675-4798-8630-338056b7370a service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:49.333036 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13557c-4675-4798-8630-338056b7370a service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:49.333414 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13557c-4675-4798-8630-338056b7370a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:49.342715 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13557c-4675-4798-8630-338056b7370a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:49.344334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13557c-4675-4798-8630-338056b7370a service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:49.353064 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1106/2232] 10.222.0.100 () {60 vars in 1389 bytes} [Tue Aug 2 09:14:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:14:59.278576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:14:59.349916 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=6 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:15:54.358173 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3410b9-37d1-419c-9fb5-a85f0b4a9077 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:15:54.371721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3410b9-37d1-419c-9fb5-a85f0b4a9077 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:15:54.378126 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356de094-ced2-4990-9382-f67c86dfb4c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:15:54.381660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356de094-ced2-4990-9382-f67c86dfb4c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:54.382195 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356de094-ced2-4990-9382-f67c86dfb4c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:54.382668 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356de094-ced2-4990-9382-f67c86dfb4c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:54.387861 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-356de094-ced2-4990-9382-f67c86dfb4c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:15:54.417226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356de094-ced2-4990-9382-f67c86dfb4c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:15:54.425469 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13557c-4675-4798-8630-338056b7370a service swift] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:15:54.432621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356de094-ced2-4990-9382-f67c86dfb4c8 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:15:54.440851 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13557c-4675-4798-8630-338056b7370a service swift] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:15:54.444864 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257bbaef-6b04-45d7-9cfa-4440bdf453a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:15:54.445982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bbaef-6b04-45d7-9cfa-4440bdf453a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:54.447267 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bbaef-6b04-45d7-9cfa-4440bdf453a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:54.447756 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bbaef-6b04-45d7-9cfa-4440bdf453a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:54.452155 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-257bbaef-6b04-45d7-9cfa-4440bdf453a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:15:54.459970 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1127/2233] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:15:54 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:15:54.474159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257bbaef-6b04-45d7-9cfa-4440bdf453a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:15:54.476205 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257bbaef-6b04-45d7-9cfa-4440bdf453a6 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:15:54.502173 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1107/2234] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:15:54 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:15:57.459131 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356de094-ced2-4990-9382-f67c86dfb4c8 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:15:57.468530 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356de094-ced2-4990-9382-f67c86dfb4c8 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:15:57.472423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4afec1-8f18-44f0-9a24-04e32ebceeeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:15:57.473041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4afec1-8f18-44f0-9a24-04e32ebceeeb service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:57.473372 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4afec1-8f18-44f0-9a24-04e32ebceeeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:57.473657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4afec1-8f18-44f0-9a24-04e32ebceeeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:57.483294 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4afec1-8f18-44f0-9a24-04e32ebceeeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:15:57.484734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4afec1-8f18-44f0-9a24-04e32ebceeeb service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:15:57.494725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1128/2235] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:15:57 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:15:58.512714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257bbaef-6b04-45d7-9cfa-4440bdf453a6 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:15:58.521477 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257bbaef-6b04-45d7-9cfa-4440bdf453a6 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:15:58.525833 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ba38e-7593-4093-9e04-37c86afd53e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:15:58.526550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ba38e-7593-4093-9e04-37c86afd53e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:58.526909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ba38e-7593-4093-9e04-37c86afd53e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:58.527261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ba38e-7593-4093-9e04-37c86afd53e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:58.536772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ba38e-7593-4093-9e04-37c86afd53e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:15:58.538331 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ba38e-7593-4093-9e04-37c86afd53e2 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:15:58.548921 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1108/2236] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:15:58 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:16:07.490193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=12 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:16:08.544334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=12 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:16:11.909381 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4afec1-8f18-44f0-9a24-04e32ebceeeb service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:16:11.924421 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4afec1-8f18-44f0-9a24-04e32ebceeeb service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:16:11.931594 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f4864-fc4d-4ea3-80eb-4ee93e880c7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:16:11.932249 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f4864-fc4d-4ea3-80eb-4ee93e880c7e service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:16:11.932542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f4864-fc4d-4ea3-80eb-4ee93e880c7e service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:16:11.932815 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f4864-fc4d-4ea3-80eb-4ee93e880c7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:16:11.934849 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-069f4864-fc4d-4ea3-80eb-4ee93e880c7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:16:11.999288 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f4864-fc4d-4ea3-80eb-4ee93e880c7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:16:12.000883 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f4864-fc4d-4ea3-80eb-4ee93e880c7e service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:16:12.010510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1129/2237] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:16:11 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:16:15.078530 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ba38e-7593-4093-9e04-37c86afd53e2 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:16:15.092176 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ba38e-7593-4093-9e04-37c86afd53e2 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:16:15.097619 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb11d9-a2d1-4b51-8225-6b7faeed9028 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:16:15.098257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb11d9-a2d1-4b51-8225-6b7faeed9028 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:16:15.098542 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb11d9-a2d1-4b51-8225-6b7faeed9028 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:16:15.098821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb11d9-a2d1-4b51-8225-6b7faeed9028 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:16:15.100813 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-85cb11d9-a2d1-4b51-8225-6b7faeed9028 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:16:15.160744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb11d9-a2d1-4b51-8225-6b7faeed9028 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:16:15.162363 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb11d9-a2d1-4b51-8225-6b7faeed9028 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:16:15.172454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1109/2238] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:16:15 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:16:15.361557 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbfc6c-d091-4b25-bb78-86ae7f0fbd8b None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:16:15.362149 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbfc6c-d091-4b25-bb78-86ae7f0fbd8b None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:16:15.362149 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbfc6c-d091-4b25-bb78-86ae7f0fbd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:16:15.382967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-c7bbfc6c-d091-4b25-bb78-86ae7f0fbd8b None None] MFA Rules not processed for user `6c30f2ab48e74d4690aafe46c4bf682f`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:16:15.384185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c7bbfc6c-d091-4b25-bb78-86ae7f0fbd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:16:15.391128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1130/2239] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:16:15 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:16:16.053033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb11d9-a2d1-4b51-8225-6b7faeed9028 service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:16:16.065591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb11d9-a2d1-4b51-8225-6b7faeed9028 service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:16:16.072467 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904454c1-8235-4d03-9cb7-c0f8afc1c85b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:16:16.073092 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904454c1-8235-4d03-9cb7-c0f8afc1c85b service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:16:16.073402 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904454c1-8235-4d03-9cb7-c0f8afc1c85b service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:16:16.073678 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904454c1-8235-4d03-9cb7-c0f8afc1c85b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:16:16.075491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-904454c1-8235-4d03-9cb7-c0f8afc1c85b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:16:16.087712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904454c1-8235-4d03-9cb7-c0f8afc1c85b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:16:16.089316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904454c1-8235-4d03-9cb7-c0f8afc1c85b service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:16:16.098681 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1110/2240] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:16:16 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:16:22.461919 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbfc6c-d091-4b25-bb78-86ae7f0fbd8b None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:16:22.472138 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbfc6c-d091-4b25-bb78-86ae7f0fbd8b None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:16:22.476508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498bc6a-8e07-44e8-8157-f5c8e09a8dfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:16:22.477176 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498bc6a-8e07-44e8-8157-f5c8e09a8dfe service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:16:22.477463 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498bc6a-8e07-44e8-8157-f5c8e09a8dfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:16:22.477733 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498bc6a-8e07-44e8-8157-f5c8e09a8dfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:16:22.479562 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-8498bc6a-8e07-44e8-8157-f5c8e09a8dfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:16:22.489715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498bc6a-8e07-44e8-8157-f5c8e09a8dfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:16:22.492273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498bc6a-8e07-44e8-8157-f5c8e09a8dfe service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:16:22.500450 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1131/2241] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:16:22 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:16:26.094398 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=34 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:16:32.496572 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=37 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:17:49.606071 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904454c1-8235-4d03-9cb7-c0f8afc1c85b service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:17:49.616685 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904454c1-8235-4d03-9cb7-c0f8afc1c85b service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:17:49.620323 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834db38f-3554-4913-9434-722b63766267 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:17:49.620936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834db38f-3554-4913-9434-722b63766267 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:17:49.621261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834db38f-3554-4913-9434-722b63766267 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:17:49.621543 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834db38f-3554-4913-9434-722b63766267 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:17:49.623293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-834db38f-3554-4913-9434-722b63766267 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:17:49.686481 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834db38f-3554-4913-9434-722b63766267 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:17:49.687950 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834db38f-3554-4913-9434-722b63766267 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:17:49.699098 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1111/2242] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:17:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:17:59.693541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=28 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:04.053575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498bc6a-8e07-44e8-8157-f5c8e09a8dfe service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:04.070544 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498bc6a-8e07-44e8-8157-f5c8e09a8dfe service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:04.076419 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a54080-a620-476f-bd58-c94150ba26a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:04.077173 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a54080-a620-476f-bd58-c94150ba26a8 service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:04.077562 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a54080-a620-476f-bd58-c94150ba26a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:04.077903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a54080-a620-476f-bd58-c94150ba26a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:04.080571 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-69a54080-a620-476f-bd58-c94150ba26a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:04.098306 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a54080-a620-476f-bd58-c94150ba26a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:04.100243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a54080-a620-476f-bd58-c94150ba26a8 service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:04.112679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1132/2243] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:18:04 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:18:14.107167 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=6 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:16.640006 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834db38f-3554-4913-9434-722b63766267 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:16.654940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834db38f-3554-4913-9434-722b63766267 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:16.661659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0484b4-bda9-40fa-8b5d-5de5e45531ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:16.662366 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0484b4-bda9-40fa-8b5d-5de5e45531ef service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:16.662790 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0484b4-bda9-40fa-8b5d-5de5e45531ef service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:16.663117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0484b4-bda9-40fa-8b5d-5de5e45531ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:16.664994 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8f0484b4-bda9-40fa-8b5d-5de5e45531ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:16.679934 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0484b4-bda9-40fa-8b5d-5de5e45531ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:16.681663 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0484b4-bda9-40fa-8b5d-5de5e45531ef service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:16.693408 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1112/2244] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:18:16 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:18:26.688606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=6 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:42.197609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a54080-a620-476f-bd58-c94150ba26a8 service nova] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:42.208885 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a54080-a620-476f-bd58-c94150ba26a8 service nova] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:42.214429 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7a090-95ea-4242-a2a7-7ef382de5593 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:42.215033 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7a090-95ea-4242-a2a7-7ef382de5593 service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:42.215322 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7a090-95ea-4242-a2a7-7ef382de5593 service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:42.215601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7a090-95ea-4242-a2a7-7ef382de5593 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:42.223873 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a7a090-95ea-4242-a2a7-7ef382de5593 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:42.225499 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a7a090-95ea-4242-a2a7-7ef382de5593 service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:42.234185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1133/2245] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:18:42 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:18:42.618924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0484b4-bda9-40fa-8b5d-5de5e45531ef service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:42.630833 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0484b4-bda9-40fa-8b5d-5de5e45531ef service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:42.636249 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac0fb5-7c0f-443c-8242-b466ac087eeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:42.636846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac0fb5-7c0f-443c-8242-b466ac087eeb service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:42.637146 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac0fb5-7c0f-443c-8242-b466ac087eeb service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:42.637425 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac0fb5-7c0f-443c-8242-b466ac087eeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:42.645137 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac0fb5-7c0f-443c-8242-b466ac087eeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:42.646703 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac0fb5-7c0f-443c-8242-b466ac087eeb service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:42.655366 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1113/2246] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:18:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:18:52.230855 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=6 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:52.651458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=6 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:53.291388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7a090-95ea-4242-a2a7-7ef382de5593 service nova] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:53.341550 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7a090-95ea-4242-a2a7-7ef382de5593 service nova] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:53.346554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7deb1-518e-44f4-9590-ba6f49c7897a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:53.347177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7deb1-518e-44f4-9590-ba6f49c7897a service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:53.347463 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7deb1-518e-44f4-9590-ba6f49c7897a service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:53.347740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7deb1-518e-44f4-9590-ba6f49c7897a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:53.372873 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7deb1-518e-44f4-9590-ba6f49c7897a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:53.374376 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7deb1-518e-44f4-9590-ba6f49c7897a service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:53.384434 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1134/2247] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:19:53 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:20:03.380148 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=24 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:20:07.281899 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac0fb5-7c0f-443c-8242-b466ac087eeb service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:07.283518 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4fac0fb5-7c0f-443c-8242-b466ac087eeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:07.364319 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac0fb5-7c0f-443c-8242-b466ac087eeb service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:07.369420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10150dc6-aa84-4c6a-b2a9-fbe1f8e07e3e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b40981fe179a43dc81553d3594dad649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:07.370044 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10150dc6-aa84-4c6a-b2a9-fbe1f8e07e3e service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:07.370322 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10150dc6-aa84-4c6a-b2a9-fbe1f8e07e3e service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:07.370595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10150dc6-aa84-4c6a-b2a9-fbe1f8e07e3e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:07.380812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10150dc6-aa84-4c6a-b2a9-fbe1f8e07e3e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:07.382451 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10150dc6-aa84-4c6a-b2a9-fbe1f8e07e3e service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:07.399061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1114/2248] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:20:07 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:20:17.388152 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=33 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:20:54.987531 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7deb1-518e-44f4-9590-ba6f49c7897a service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:55.007531 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7deb1-518e-44f4-9590-ba6f49c7897a service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:55.011222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eee97c-5eb1-43ee-bdaf-e36bd76f0fa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:55.011830 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eee97c-5eb1-43ee-bdaf-e36bd76f0fa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:55.012171 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eee97c-5eb1-43ee-bdaf-e36bd76f0fa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:55.012447 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eee97c-5eb1-43ee-bdaf-e36bd76f0fa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:55.030688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eee97c-5eb1-43ee-bdaf-e36bd76f0fa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:55.032168 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eee97c-5eb1-43ee-bdaf-e36bd76f0fa5 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:55.054870 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1135/2249] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:20:54 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:20:55.183040 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10150dc6-aa84-4c6a-b2a9-fbe1f8e07e3e service placement] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:55.193705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10150dc6-aa84-4c6a-b2a9-fbe1f8e07e3e service placement] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:55.197531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5e08d-0dfb-4af5-b8df-7a77a8a9791e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:55.198224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5e08d-0dfb-4af5-b8df-7a77a8a9791e service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:55.198575 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5e08d-0dfb-4af5-b8df-7a77a8a9791e service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:55.198919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5e08d-0dfb-4af5-b8df-7a77a8a9791e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:55.211228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af5e08d-0dfb-4af5-b8df-7a77a8a9791e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:55.212672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af5e08d-0dfb-4af5-b8df-7a77a8a9791e service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:55.224641 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1115/2250] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:20:55 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:21:05.052155 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=17 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:21:05.219497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=6 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:21:12.241507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eee97c-5eb1-43ee-bdaf-e36bd76f0fa5 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:21:12.254652 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eee97c-5eb1-43ee-bdaf-e36bd76f0fa5 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:21:12.260141 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad6942-0b8a-4dc6-a217-022c2aa2f30a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:21:12.260772 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad6942-0b8a-4dc6-a217-022c2aa2f30a service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:21:12.261056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad6942-0b8a-4dc6-a217-022c2aa2f30a service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:21:12.261451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad6942-0b8a-4dc6-a217-022c2aa2f30a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:21:12.269315 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad6942-0b8a-4dc6-a217-022c2aa2f30a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:21:12.270900 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad6942-0b8a-4dc6-a217-022c2aa2f30a service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:21:12.298212 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1136/2251] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:21:12 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:21:15.211579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5e08d-0dfb-4af5-b8df-7a77a8a9791e service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:21:15.223410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5e08d-0dfb-4af5-b8df-7a77a8a9791e service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:21:15.228831 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0db25e-128c-48cc-8efe-2953ecb43ced service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:21:15.229476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0db25e-128c-48cc-8efe-2953ecb43ced service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:21:15.229765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0db25e-128c-48cc-8efe-2953ecb43ced service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:21:15.230594 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0db25e-128c-48cc-8efe-2953ecb43ced service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:21:15.238678 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0db25e-128c-48cc-8efe-2953ecb43ced service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:21:15.240281 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0db25e-128c-48cc-8efe-2953ecb43ced service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:21:15.267752 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1116/2252] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:21:15 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:21:15.947596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad6942-0b8a-4dc6-a217-022c2aa2f30a service nova] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:21:15.959653 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad6942-0b8a-4dc6-a217-022c2aa2f30a service nova] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:21:15.965266 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7ccdd-3f5e-4dff-9000-6e3824c28664 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:21:15.965882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7ccdd-3f5e-4dff-9000-6e3824c28664 service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:21:15.966167 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7ccdd-3f5e-4dff-9000-6e3824c28664 service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:21:15.966442 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7ccdd-3f5e-4dff-9000-6e3824c28664 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:21:15.975264 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7ccdd-3f5e-4dff-9000-6e3824c28664 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:21:15.976873 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7ccdd-3f5e-4dff-9000-6e3824c28664 service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:21:16.004254 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1137/2253] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:21:15 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:21:25.265135 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=13 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:21:26.001670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=26 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:22:03.907193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0db25e-128c-48cc-8efe-2953ecb43ced service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:22:03.916788 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0db25e-128c-48cc-8efe-2953ecb43ced service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:22:03.925609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e0444-2b9b-43da-bb49-75c5a3029b88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:22:03.926257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e0444-2b9b-43da-bb49-75c5a3029b88 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:22:03.926545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e0444-2b9b-43da-bb49-75c5a3029b88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:22:03.926827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e0444-2b9b-43da-bb49-75c5a3029b88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:22:03.936595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e0444-2b9b-43da-bb49-75c5a3029b88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:22:03.938102 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e0444-2b9b-43da-bb49-75c5a3029b88 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:22:03.946902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1117/2254] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:22:03 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:22:13.943015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=6 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:53.599735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7ccdd-3f5e-4dff-9000-6e3824c28664 service nova] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:53.602026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-fea7ccdd-3f5e-4dff-9000-6e3824c28664 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:53.672888 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7ccdd-3f5e-4dff-9000-6e3824c28664 service nova] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:53.676488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ed521-936d-46ee-9d37-ea1c59808765 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:53.677097 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ed521-936d-46ee-9d37-ea1c59808765 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:53.677409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ed521-936d-46ee-9d37-ea1c59808765 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:53.677688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ed521-936d-46ee-9d37-ea1c59808765 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:53.679543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e98ed521-936d-46ee-9d37-ea1c59808765 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:53.771391 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ed521-936d-46ee-9d37-ea1c59808765 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:53.772846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ed521-936d-46ee-9d37-ea1c59808765 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:53.792154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1138/2255] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:23:53 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:24:03.789484 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=66 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:24:04.847177 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e0444-2b9b-43da-bb49-75c5a3029b88 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:24:04.848621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-348e0444-2b9b-43da-bb49-75c5a3029b88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:24:04.861408 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e0444-2b9b-43da-bb49-75c5a3029b88 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:24:04.865372 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829329b1-f67e-4834-9698-4f539d1ea813 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:24:04.866240 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829329b1-f67e-4834-9698-4f539d1ea813 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:24:04.866535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829329b1-f67e-4834-9698-4f539d1ea813 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:24:04.866822 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829329b1-f67e-4834-9698-4f539d1ea813 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:24:04.879971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829329b1-f67e-4834-9698-4f539d1ea813 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:24:04.882255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829329b1-f67e-4834-9698-4f539d1ea813 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:24:04.913549 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1118/2256] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:24:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:24:05.307906 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ed521-936d-46ee-9d37-ea1c59808765 service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:24:05.317908 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ed521-936d-46ee-9d37-ea1c59808765 service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:24:05.327914 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620a0eb-fbdc-4776-8aae-c76b9ba4fc40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:24:05.327914 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a0eb-fbdc-4776-8aae-c76b9ba4fc40 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:24:05.327914 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a0eb-fbdc-4776-8aae-c76b9ba4fc40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:24:05.327914 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a0eb-fbdc-4776-8aae-c76b9ba4fc40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:24:05.328398 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-5620a0eb-fbdc-4776-8aae-c76b9ba4fc40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:24:05.415617 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620a0eb-fbdc-4776-8aae-c76b9ba4fc40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:24:05.417104 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620a0eb-fbdc-4776-8aae-c76b9ba4fc40 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:24:05.446896 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1139/2257] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:24:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:24:14.910205 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=13 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:24:15.443965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=41 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:25:13.260968 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829329b1-f67e-4834-9698-4f539d1ea813 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:25:13.262705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-829329b1-f67e-4834-9698-4f539d1ea813 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:25:13.339700 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829329b1-f67e-4834-9698-4f539d1ea813 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:25:13.345594 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbca38-641e-4816-895f-1b620981aa80 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b40981fe179a43dc81553d3594dad649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:25:13.346256 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbca38-641e-4816-895f-1b620981aa80 service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:25:13.346545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbca38-641e-4816-895f-1b620981aa80 service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:25:13.346830 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbca38-641e-4816-895f-1b620981aa80 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:25:13.348738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-10fbca38-641e-4816-895f-1b620981aa80 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:25:13.361969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fbca38-641e-4816-895f-1b620981aa80 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:25:13.363579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fbca38-641e-4816-895f-1b620981aa80 service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:25:13.377134 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1119/2258] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:25:13 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:25:23.372219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=32 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:26:05.527205 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620a0eb-fbdc-4776-8aae-c76b9ba4fc40 service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:26:05.537369 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620a0eb-fbdc-4776-8aae-c76b9ba4fc40 service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:26:05.541035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e6f20-0b42-4f9b-8845-b2e96325f8b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:26:05.541680 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e6f20-0b42-4f9b-8845-b2e96325f8b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:26:05.541983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e6f20-0b42-4f9b-8845-b2e96325f8b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:26:05.542259 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e6f20-0b42-4f9b-8845-b2e96325f8b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:26:05.544174 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-8d3e6f20-0b42-4f9b-8845-b2e96325f8b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:26:05.644225 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e6f20-0b42-4f9b-8845-b2e96325f8b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:26:05.645709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e6f20-0b42-4f9b-8845-b2e96325f8b5 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:26:05.657672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1140/2259] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:26:05 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:26:05.664779 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbca38-641e-4816-895f-1b620981aa80 service placement] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:26:05.674739 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbca38-641e-4816-895f-1b620981aa80 service placement] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:26:05.681155 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c274f80-2760-40b3-824f-81eab5727067 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:26:05.681805 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c274f80-2760-40b3-824f-81eab5727067 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:26:05.682065 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c274f80-2760-40b3-824f-81eab5727067 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:26:05.682347 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c274f80-2760-40b3-824f-81eab5727067 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:26:05.684414 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3c274f80-2760-40b3-824f-81eab5727067 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:26:05.771529 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c274f80-2760-40b3-824f-81eab5727067 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:26:05.773072 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c274f80-2760-40b3-824f-81eab5727067 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:26:05.792075 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1120/2260] 10.222.0.100 () {60 vars in 1401 bytes} [Tue Aug 2 09:26:05 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:26:05.839569 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e6f20-0b42-4f9b-8845-b2e96325f8b5 service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:26:05.849883 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e6f20-0b42-4f9b-8845-b2e96325f8b5 service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:26:05.853824 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5904bf-8c46-49e3-853f-aba13daef03d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:26:05.854444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5904bf-8c46-49e3-853f-aba13daef03d service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:26:05.854729 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5904bf-8c46-49e3-853f-aba13daef03d service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:26:05.855008 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5904bf-8c46-49e3-853f-aba13daef03d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:26:05.856873 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-5c5904bf-8c46-49e3-853f-aba13daef03d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:26:05.870927 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5904bf-8c46-49e3-853f-aba13daef03d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:26:05.872350 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5904bf-8c46-49e3-853f-aba13daef03d service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:26:05.885318 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1141/2261] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:26:05 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:26:15.789276 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=41 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:26:15.879293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=45 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:27:07.900411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c274f80-2760-40b3-824f-81eab5727067 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:27:07.902082 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3c274f80-2760-40b3-824f-81eab5727067 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:27:07.971468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c274f80-2760-40b3-824f-81eab5727067 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:27:07.975065 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8549e9a-8fd6-472c-9384-14402fd8bc2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:27:07.975688 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8549e9a-8fd6-472c-9384-14402fd8bc2b service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:27:07.975972 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8549e9a-8fd6-472c-9384-14402fd8bc2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:27:07.976252 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8549e9a-8fd6-472c-9384-14402fd8bc2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:27:07.978149 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d8549e9a-8fd6-472c-9384-14402fd8bc2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:27:07.987492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8549e9a-8fd6-472c-9384-14402fd8bc2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:27:07.988905 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8549e9a-8fd6-472c-9384-14402fd8bc2b service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:27:07.996652 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1121/2262] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:27:07 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:27:17.993301 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=31 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:27:53.955976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5904bf-8c46-49e3-853f-aba13daef03d service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:27:53.964520 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5904bf-8c46-49e3-853f-aba13daef03d service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:27:53.968353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6908a78-534b-433e-867c-ce06371a8903 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:27:53.968956 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6908a78-534b-433e-867c-ce06371a8903 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:27:53.969273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6908a78-534b-433e-867c-ce06371a8903 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:27:53.969553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6908a78-534b-433e-867c-ce06371a8903 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:27:53.971424 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b6908a78-534b-433e-867c-ce06371a8903 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:27:54.045983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6908a78-534b-433e-867c-ce06371a8903 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:27:54.047419 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6908a78-534b-433e-867c-ce06371a8903 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:27:54.056678 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1142/2263] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:27:53 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:27:54.104572 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8549e9a-8fd6-472c-9384-14402fd8bc2b service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:27:54.114413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8549e9a-8fd6-472c-9384-14402fd8bc2b service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:27:54.117956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b861199-6d3c-47bb-834f-e9a2913ead4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:27:54.118553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b861199-6d3c-47bb-834f-e9a2913ead4f service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:27:54.118833 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b861199-6d3c-47bb-834f-e9a2913ead4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:27:54.119128 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b861199-6d3c-47bb-834f-e9a2913ead4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:27:54.120942 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4b861199-6d3c-47bb-834f-e9a2913ead4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:27:54.194091 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b861199-6d3c-47bb-834f-e9a2913ead4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:27:54.195458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b861199-6d3c-47bb-834f-e9a2913ead4f service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:27:54.204998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1122/2264] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:27:54 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:28:04.052226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=33 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:28:04.200933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=33 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:28:06.248884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6908a78-534b-433e-867c-ce06371a8903 service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:28:06.257069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6908a78-534b-433e-867c-ce06371a8903 service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:28:06.260876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd2733-4ba3-46da-9cf9-db5543901424 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:28:06.261510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd2733-4ba3-46da-9cf9-db5543901424 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:28:06.261790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd2733-4ba3-46da-9cf9-db5543901424 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:28:06.262060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd2733-4ba3-46da-9cf9-db5543901424 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:28:06.264589 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-a7cd2733-4ba3-46da-9cf9-db5543901424 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:28:06.334720 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd2733-4ba3-46da-9cf9-db5543901424 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:28:06.336638 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd2733-4ba3-46da-9cf9-db5543901424 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:28:06.346465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1143/2265] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:28:06 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:28:16.342314 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=33 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:29:54.406324 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b861199-6d3c-47bb-834f-e9a2913ead4f service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:29:54.433150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b861199-6d3c-47bb-834f-e9a2913ead4f service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:29:54.436637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f1291-77bd-4548-9c0e-344ac3b0ba62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:29:54.437331 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f1291-77bd-4548-9c0e-344ac3b0ba62 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:29:54.437613 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f1291-77bd-4548-9c0e-344ac3b0ba62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:29:54.437890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f1291-77bd-4548-9c0e-344ac3b0ba62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:29:54.439726 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e82f1291-77bd-4548-9c0e-344ac3b0ba62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:29:54.509978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82f1291-77bd-4548-9c0e-344ac3b0ba62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:29:54.511407 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82f1291-77bd-4548-9c0e-344ac3b0ba62 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:29:54.520316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1123/2266] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:29:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:29:54.567525 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd2733-4ba3-46da-9cf9-db5543901424 service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:29:54.586113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd2733-4ba3-46da-9cf9-db5543901424 service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:29:54.589840 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f836eba-66e0-4d4c-93f7-2e2ddb091495 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:29:54.590453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f836eba-66e0-4d4c-93f7-2e2ddb091495 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:29:54.590737 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f836eba-66e0-4d4c-93f7-2e2ddb091495 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:29:54.591012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f836eba-66e0-4d4c-93f7-2e2ddb091495 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:29:54.599433 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f836eba-66e0-4d4c-93f7-2e2ddb091495 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:29:54.600946 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f836eba-66e0-4d4c-93f7-2e2ddb091495 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:29:54.610677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1144/2267] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:29:54 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:30:04.516226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=40 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:30:04.605926 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=10 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:30:06.707172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f1291-77bd-4548-9c0e-344ac3b0ba62 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:30:06.707172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e82f1291-77bd-4548-9c0e-344ac3b0ba62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:30:06.723577 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f1291-77bd-4548-9c0e-344ac3b0ba62 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:30:06.730587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5679a5a-b077-40b7-8542-b1d74124fefe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:30:06.731284 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5679a5a-b077-40b7-8542-b1d74124fefe service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:30:06.731565 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5679a5a-b077-40b7-8542-b1d74124fefe service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:30:06.731845 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5679a5a-b077-40b7-8542-b1d74124fefe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:30:06.748634 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5679a5a-b077-40b7-8542-b1d74124fefe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:30:06.750544 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5679a5a-b077-40b7-8542-b1d74124fefe service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:30:06.765139 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1124/2268] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:30:06 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:30:16.755954 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=6 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:30:20.209517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f836eba-66e0-4d4c-93f7-2e2ddb091495 service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:30:20.211069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-5f836eba-66e0-4d4c-93f7-2e2ddb091495 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:30:20.286889 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f836eba-66e0-4d4c-93f7-2e2ddb091495 service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:30:20.292462 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c6d07-8b6c-434d-bf07-595b7039f913 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b40981fe179a43dc81553d3594dad649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:30:20.293070 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c6d07-8b6c-434d-bf07-595b7039f913 service placement] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:30:20.293373 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c6d07-8b6c-434d-bf07-595b7039f913 service placement] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:30:20.293645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c6d07-8b6c-434d-bf07-595b7039f913 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:30:20.302160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24c6d07-8b6c-434d-bf07-595b7039f913 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:30:20.303726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24c6d07-8b6c-434d-bf07-595b7039f913 service placement] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:30:20.314178 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1145/2269] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:30:20 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:30:30.310004 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=33 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:32:06.899009 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5679a5a-b077-40b7-8542-b1d74124fefe service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:32:06.920169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5679a5a-b077-40b7-8542-b1d74124fefe service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:32:06.924034 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634dff2-f840-452d-a7f6-3c32952867a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:32:06.924649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634dff2-f840-452d-a7f6-3c32952867a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:32:06.924934 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634dff2-f840-452d-a7f6-3c32952867a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:32:06.925240 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634dff2-f840-452d-a7f6-3c32952867a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:32:06.943224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634dff2-f840-452d-a7f6-3c32952867a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:32:06.944636 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634dff2-f840-452d-a7f6-3c32952867a1 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:32:06.966394 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1125/2270] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:32:06 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:32:06.973369 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c6d07-8b6c-434d-bf07-595b7039f913 service placement] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:32:06.983644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c6d07-8b6c-434d-bf07-595b7039f913 service placement] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:32:06.987248 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4780b-225d-4361-b113-1cf00e29ca0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:32:06.987859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4780b-225d-4361-b113-1cf00e29ca0b service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:32:06.988143 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4780b-225d-4361-b113-1cf00e29ca0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:32:06.988421 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4780b-225d-4361-b113-1cf00e29ca0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:32:06.997414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4780b-225d-4361-b113-1cf00e29ca0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:32:06.998840 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4780b-225d-4361-b113-1cf00e29ca0b service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:32:07.007510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1146/2271] 10.222.0.100 () {60 vars in 1401 bytes} [Tue Aug 2 09:32:06 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:32:07.057673 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634dff2-f840-452d-a7f6-3c32952867a1 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:32:07.067240 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634dff2-f840-452d-a7f6-3c32952867a1 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:32:07.070887 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042f700-aa98-4722-904f-2c16d6ce8aeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:32:07.071498 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042f700-aa98-4722-904f-2c16d6ce8aeb service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:32:07.071832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042f700-aa98-4722-904f-2c16d6ce8aeb service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:32:07.072108 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042f700-aa98-4722-904f-2c16d6ce8aeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:32:07.083948 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b042f700-aa98-4722-904f-2c16d6ce8aeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:32:07.085388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b042f700-aa98-4722-904f-2c16d6ce8aeb service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:32:07.096837 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1126/2272] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:32:07 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:32:14.923798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4780b-225d-4361-b113-1cf00e29ca0b service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:32:14.945374 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4780b-225d-4361-b113-1cf00e29ca0b service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:32:14.951849 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c601dc-b722-44d0-932e-b431f3ff5874 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:32:14.952467 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c601dc-b722-44d0-932e-b431f3ff5874 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:32:14.952752 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c601dc-b722-44d0-932e-b431f3ff5874 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:32:14.953034 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c601dc-b722-44d0-932e-b431f3ff5874 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:32:14.960721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c601dc-b722-44d0-932e-b431f3ff5874 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:32:14.962238 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c601dc-b722-44d0-932e-b431f3ff5874 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:32:14.970879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1147/2273] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:32:14 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:32:17.092100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=24 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:32:24.967527 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=16 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:33:54.875698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042f700-aa98-4722-904f-2c16d6ce8aeb service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:33:54.878988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b042f700-aa98-4722-904f-2c16d6ce8aeb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:33:54.975119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042f700-aa98-4722-904f-2c16d6ce8aeb service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:33:54.980235 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c7220-9184-47b4-aa86-5742acf2d297 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:33:54.980933 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c7220-9184-47b4-aa86-5742acf2d297 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:33:54.981312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c7220-9184-47b4-aa86-5742acf2d297 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:33:54.981654 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c7220-9184-47b4-aa86-5742acf2d297 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:33:54.991215 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c7220-9184-47b4-aa86-5742acf2d297 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:33:54.993863 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c7220-9184-47b4-aa86-5742acf2d297 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:33:55.022745 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1127/2274] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:33:54 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:33:55.073845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c601dc-b722-44d0-932e-b431f3ff5874 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:33:55.083368 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c601dc-b722-44d0-932e-b431f3ff5874 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:33:55.087287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c074b-803a-46bb-9f53-7ab71c85c21b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:33:55.087905 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c074b-803a-46bb-9f53-7ab71c85c21b service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:33:55.088192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c074b-803a-46bb-9f53-7ab71c85c21b service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:33:55.088469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c074b-803a-46bb-9f53-7ab71c85c21b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:33:55.097536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c074b-803a-46bb-9f53-7ab71c85c21b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:33:55.098965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c074b-803a-46bb-9f53-7ab71c85c21b service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:33:55.128459 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1148/2275] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:33:55 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:33:55.223335 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c7220-9184-47b4-aa86-5742acf2d297 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:33:55.224905 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b22c7220-9184-47b4-aa86-5742acf2d297 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:33:55.240388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c7220-9184-47b4-aa86-5742acf2d297 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:33:55.246518 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11cd923-f352-493d-b463-422854384ccb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:33:55.247131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cd923-f352-493d-b463-422854384ccb service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:33:55.247413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cd923-f352-493d-b463-422854384ccb service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:33:55.247687 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cd923-f352-493d-b463-422854384ccb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:33:55.255713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11cd923-f352-493d-b463-422854384ccb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:33:55.257773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11cd923-f352-493d-b463-422854384ccb service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:33:55.298271 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1128/2276] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:33:55 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:34:05.125338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=13 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:34:05.295275 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=51 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:35:24.180460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c074b-803a-46bb-9f53-7ab71c85c21b service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:35:24.246941 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c074b-803a-46bb-9f53-7ab71c85c21b service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:35:24.251887 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950daa0-2ee8-4bac-b910-27de1a2bca85 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b40981fe179a43dc81553d3594dad649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:35:24.252499 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950daa0-2ee8-4bac-b910-27de1a2bca85 service placement] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:35:24.252785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950daa0-2ee8-4bac-b910-27de1a2bca85 service placement] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:35:24.253093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950daa0-2ee8-4bac-b910-27de1a2bca85 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:35:24.254990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-8950daa0-2ee8-4bac-b910-27de1a2bca85 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:35:24.266639 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950daa0-2ee8-4bac-b910-27de1a2bca85 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:35:24.268220 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950daa0-2ee8-4bac-b910-27de1a2bca85 service placement] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:35:24.278491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1149/2277] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:35:24 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:35:34.274379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=32 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:35:55.378999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11cd923-f352-493d-b463-422854384ccb service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:35:55.389823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11cd923-f352-493d-b463-422854384ccb service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:35:55.395417 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf50b67-bdf8-47f2-a348-808e40b814a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:35:55.396518 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf50b67-bdf8-47f2-a348-808e40b814a3 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:35:55.396518 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf50b67-bdf8-47f2-a348-808e40b814a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:35:55.396738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf50b67-bdf8-47f2-a348-808e40b814a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:35:55.399239 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9cf50b67-bdf8-47f2-a348-808e40b814a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:35:55.430115 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950daa0-2ee8-4bac-b910-27de1a2bca85 service placement] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:35:55.442379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950daa0-2ee8-4bac-b910-27de1a2bca85 service placement] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:35:55.448324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44af9f-7a2a-4c7c-8428-7fbbf57209a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:35:55.449057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44af9f-7a2a-4c7c-8428-7fbbf57209a9 service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:35:55.449449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44af9f-7a2a-4c7c-8428-7fbbf57209a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:35:55.449787 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44af9f-7a2a-4c7c-8428-7fbbf57209a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:35:55.451692 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-4a44af9f-7a2a-4c7c-8428-7fbbf57209a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:35:55.498528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf50b67-bdf8-47f2-a348-808e40b814a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:35:55.502835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf50b67-bdf8-47f2-a348-808e40b814a3 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:35:55.531172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1129/2278] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:35:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:35:55.544607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44af9f-7a2a-4c7c-8428-7fbbf57209a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:35:55.545410 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44af9f-7a2a-4c7c-8428-7fbbf57209a9 service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:35:55.566048 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1150/2279] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:35:55 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:35:55.706790 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf50b67-bdf8-47f2-a348-808e40b814a3 service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:35:55.723279 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf50b67-bdf8-47f2-a348-808e40b814a3 service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:35:55.728972 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a72d0d-5aa5-4684-8b8c-3490c984277d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:35:55.730147 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a72d0d-5aa5-4684-8b8c-3490c984277d service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:35:55.730501 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a72d0d-5aa5-4684-8b8c-3490c984277d service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:35:55.730885 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a72d0d-5aa5-4684-8b8c-3490c984277d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:35:55.740320 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a72d0d-5aa5-4684-8b8c-3490c984277d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:35:55.742582 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a72d0d-5aa5-4684-8b8c-3490c984277d service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:35:55.776817 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1130/2280] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:35:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:36:05.562730 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=41 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:36:05.773337 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=54 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:37:19.878511 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44af9f-7a2a-4c7c-8428-7fbbf57209a9 service nova] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:37:19.880232 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-4a44af9f-7a2a-4c7c-8428-7fbbf57209a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:37:19.951070 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44af9f-7a2a-4c7c-8428-7fbbf57209a9 service nova] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:37:19.955398 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f8cf2-2f7b-40af-a32b-fbd3210a2b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:37:19.956013 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f8cf2-2f7b-40af-a32b-fbd3210a2b0f service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:37:19.956297 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f8cf2-2f7b-40af-a32b-fbd3210a2b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:37:19.956575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f8cf2-2f7b-40af-a32b-fbd3210a2b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:37:19.958444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-832f8cf2-2f7b-40af-a32b-fbd3210a2b0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:37:19.967748 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f8cf2-2f7b-40af-a32b-fbd3210a2b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:37:19.969199 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f8cf2-2f7b-40af-a32b-fbd3210a2b0f service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:37:19.977017 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1151/2281] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:37:19 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:37:29.973548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=31 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:37:55.812624 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a72d0d-5aa5-4684-8b8c-3490c984277d service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:37:55.821317 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a72d0d-5aa5-4684-8b8c-3490c984277d service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:37:55.824854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c33e11-d168-4c3d-94f4-4b1d62c2d439 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:37:55.825495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c33e11-d168-4c3d-94f4-4b1d62c2d439 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:37:55.825882 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c33e11-d168-4c3d-94f4-4b1d62c2d439 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:37:55.826164 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c33e11-d168-4c3d-94f4-4b1d62c2d439 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:37:55.827986 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c8c33e11-d168-4c3d-94f4-4b1d62c2d439 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:37:55.911868 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c33e11-d168-4c3d-94f4-4b1d62c2d439 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:37:55.913342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c33e11-d168-4c3d-94f4-4b1d62c2d439 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:37:55.924965 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1131/2282] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:37:55 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:37:55.932149 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f8cf2-2f7b-40af-a32b-fbd3210a2b0f service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:37:55.941217 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f8cf2-2f7b-40af-a32b-fbd3210a2b0f service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:37:55.944762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfa8a2-0548-4338-ad64-ffb0b864bac9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:37:55.945396 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfa8a2-0548-4338-ad64-ffb0b864bac9 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:37:55.945675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfa8a2-0548-4338-ad64-ffb0b864bac9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:37:55.945946 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfa8a2-0548-4338-ad64-ffb0b864bac9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:37:55.947782 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-0cbfa8a2-0548-4338-ad64-ffb0b864bac9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:37:56.026625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfa8a2-0548-4338-ad64-ffb0b864bac9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:37:56.028188 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfa8a2-0548-4338-ad64-ffb0b864bac9 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:37:56.046738 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1152/2283] 10.222.0.100 () {60 vars in 1401 bytes} [Tue Aug 2 09:37:55 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:37:56.088972 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c33e11-d168-4c3d-94f4-4b1d62c2d439 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:37:56.098001 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c33e11-d168-4c3d-94f4-4b1d62c2d439 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:37:56.101833 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07555b11-4c4d-4559-82b0-ed8b9f3f7585 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:37:56.102395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07555b11-4c4d-4559-82b0-ed8b9f3f7585 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:37:56.102681 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07555b11-4c4d-4559-82b0-ed8b9f3f7585 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:37:56.103023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07555b11-4c4d-4559-82b0-ed8b9f3f7585 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:37:56.104812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-07555b11-4c4d-4559-82b0-ed8b9f3f7585 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:37:56.118670 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07555b11-4c4d-4559-82b0-ed8b9f3f7585 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:37:56.120097 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07555b11-4c4d-4559-82b0-ed8b9f3f7585 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:37:56.131966 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1132/2284] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:37:56 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:37:56.966854 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfa8a2-0548-4338-ad64-ffb0b864bac9 service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:37:56.979753 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfa8a2-0548-4338-ad64-ffb0b864bac9 service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:37:56.985297 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a3d07-33be-4edc-9c75-8d89a16fddae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:37:56.986227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a3d07-33be-4edc-9c75-8d89a16fddae service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:37:56.987227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a3d07-33be-4edc-9c75-8d89a16fddae service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:37:56.987618 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a3d07-33be-4edc-9c75-8d89a16fddae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:37:56.989604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-232a3d07-33be-4edc-9c75-8d89a16fddae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:37:57.002652 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a3d07-33be-4edc-9c75-8d89a16fddae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:37:57.004227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a3d07-33be-4edc-9c75-8d89a16fddae service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:37:57.015103 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1153/2285] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:37:56 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:38:06.126551 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=45 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:38:07.010658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=47 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:38:54.363404 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07555b11-4c4d-4559-82b0-ed8b9f3f7585 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:38:54.375335 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07555b11-4c4d-4559-82b0-ed8b9f3f7585 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:38:54.380671 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ac59a-5f54-443e-a7cb-8e7fedeb7165 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:38:54.381318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ac59a-5f54-443e-a7cb-8e7fedeb7165 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:38:54.381598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ac59a-5f54-443e-a7cb-8e7fedeb7165 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:38:54.381871 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ac59a-5f54-443e-a7cb-8e7fedeb7165 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:38:54.383665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8c1ac59a-5f54-443e-a7cb-8e7fedeb7165 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:38:54.454091 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ac59a-5f54-443e-a7cb-8e7fedeb7165 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:38:54.455725 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ac59a-5f54-443e-a7cb-8e7fedeb7165 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:38:54.464641 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1133/2286] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:38:54 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:38:54.624583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a3d07-33be-4edc-9c75-8d89a16fddae service nova] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:38:54.636092 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a3d07-33be-4edc-9c75-8d89a16fddae service nova] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:38:54.641571 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ce1c9-6681-4ca0-8a7c-76de377a2577 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:38:54.642308 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ce1c9-6681-4ca0-8a7c-76de377a2577 service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:38:54.642597 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ce1c9-6681-4ca0-8a7c-76de377a2577 service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:38:54.642876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ce1c9-6681-4ca0-8a7c-76de377a2577 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:38:54.644722 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-566ce1c9-6681-4ca0-8a7c-76de377a2577 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:38:54.715109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ce1c9-6681-4ca0-8a7c-76de377a2577 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:38:54.716742 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ce1c9-6681-4ca0-8a7c-76de377a2577 service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:38:54.725803 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1154/2287] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:38:54 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:38:54.883375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ac59a-5f54-443e-a7cb-8e7fedeb7165 service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:38:54.894557 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ac59a-5f54-443e-a7cb-8e7fedeb7165 service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:38:54.900016 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db61b01-0c7f-45c5-a4b8-669defb40fff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:38:54.900629 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db61b01-0c7f-45c5-a4b8-669defb40fff service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:38:54.900908 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db61b01-0c7f-45c5-a4b8-669defb40fff service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:38:54.901223 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db61b01-0c7f-45c5-a4b8-669defb40fff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:38:54.903834 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9db61b01-0c7f-45c5-a4b8-669defb40fff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:38:54.974889 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db61b01-0c7f-45c5-a4b8-669defb40fff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:38:54.976513 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db61b01-0c7f-45c5-a4b8-669defb40fff service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:38:54.987604 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1134/2288] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:38:54 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:39:04.721609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=33 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:39:04.982918 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=66 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:40:29.178458 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ce1c9-6681-4ca0-8a7c-76de377a2577 service nova] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:40:29.180046 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-566ce1c9-6681-4ca0-8a7c-76de377a2577 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:40:29.255282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ce1c9-6681-4ca0-8a7c-76de377a2577 service nova] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:40:29.260519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bafed2-cdb7-47e0-b3aa-837bdf42dbdd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b40981fe179a43dc81553d3594dad649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:40:29.261147 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bafed2-cdb7-47e0-b3aa-837bdf42dbdd service placement] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:40:29.262055 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bafed2-cdb7-47e0-b3aa-837bdf42dbdd service placement] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:40:29.262055 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bafed2-cdb7-47e0-b3aa-837bdf42dbdd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:40:29.270807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bafed2-cdb7-47e0-b3aa-837bdf42dbdd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:40:29.272382 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bafed2-cdb7-47e0-b3aa-837bdf42dbdd service placement] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:40:29.283140 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1155/2289] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:40:29 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:40:39.278924 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=32 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:40:55.559082 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db61b01-0c7f-45c5-a4b8-669defb40fff service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:40:55.591389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db61b01-0c7f-45c5-a4b8-669defb40fff service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:40:55.597024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885e76d-8dda-4ecf-9a04-372af20a47f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:40:55.597670 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885e76d-8dda-4ecf-9a04-372af20a47f0 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:40:55.597954 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885e76d-8dda-4ecf-9a04-372af20a47f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:40:55.598231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885e76d-8dda-4ecf-9a04-372af20a47f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:40:55.606642 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885e76d-8dda-4ecf-9a04-372af20a47f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:40:55.608209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885e76d-8dda-4ecf-9a04-372af20a47f0 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:40:55.617389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1135/2290] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:40:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:41:05.612945 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=15 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:41:56.557261 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bafed2-cdb7-47e0-b3aa-837bdf42dbdd service placement] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:41:56.576308 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bafed2-cdb7-47e0-b3aa-837bdf42dbdd service placement] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:41:56.579929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de1ed1-6373-4483-b090-bf5ab98fcef2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:41:56.580527 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de1ed1-6373-4483-b090-bf5ab98fcef2 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:41:56.580806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de1ed1-6373-4483-b090-bf5ab98fcef2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:41:56.581079 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de1ed1-6373-4483-b090-bf5ab98fcef2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:41:56.582921 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d4de1ed1-6373-4483-b090-bf5ab98fcef2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:41:56.653105 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de1ed1-6373-4483-b090-bf5ab98fcef2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:41:56.654547 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de1ed1-6373-4483-b090-bf5ab98fcef2 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:41:56.663368 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1156/2291] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:41:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:42:06.659194 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=37 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:42:08.553380 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885e76d-8dda-4ecf-9a04-372af20a47f0 service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:42:08.555011 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e885e76d-8dda-4ecf-9a04-372af20a47f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:42:08.577617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885e76d-8dda-4ecf-9a04-372af20a47f0 service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:42:08.581329 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e1cd9-5b89-472b-b8b7-59fb69c0b7ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:42:08.581976 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e1cd9-5b89-472b-b8b7-59fb69c0b7ef service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:42:08.582268 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e1cd9-5b89-472b-b8b7-59fb69c0b7ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:42:08.582550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e1cd9-5b89-472b-b8b7-59fb69c0b7ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:42:08.591885 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e1cd9-5b89-472b-b8b7-59fb69c0b7ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:42:08.593372 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e1cd9-5b89-472b-b8b7-59fb69c0b7ef service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:42:08.603209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1136/2292] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:42:08 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:42:18.598265 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=11 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:42:24.850972 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de1ed1-6373-4483-b090-bf5ab98fcef2 service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:42:24.879890 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de1ed1-6373-4483-b090-bf5ab98fcef2 service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:42:24.884534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce37651-d813-46fd-9d41-eb436495a1ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:42:24.885177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce37651-d813-46fd-9d41-eb436495a1ee service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:42:24.885473 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce37651-d813-46fd-9d41-eb436495a1ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:42:24.885754 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce37651-d813-46fd-9d41-eb436495a1ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:42:24.894419 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce37651-d813-46fd-9d41-eb436495a1ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:42:24.896648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce37651-d813-46fd-9d41-eb436495a1ee service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:42:24.906539 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1157/2293] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:42:24 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:42:34.901935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=15 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:42:55.856791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e1cd9-5b89-472b-b8b7-59fb69c0b7ef service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:42:55.858494 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-af9e1cd9-5b89-472b-b8b7-59fb69c0b7ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:42:55.870480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e1cd9-5b89-472b-b8b7-59fb69c0b7ef service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:42:55.874155 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8021e5-e48d-4c08-a432-d47582ad80d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:42:55.874763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8021e5-e48d-4c08-a432-d47582ad80d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:42:55.875051 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8021e5-e48d-4c08-a432-d47582ad80d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:42:55.875332 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8021e5-e48d-4c08-a432-d47582ad80d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:42:55.893142 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8021e5-e48d-4c08-a432-d47582ad80d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:42:55.894558 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8021e5-e48d-4c08-a432-d47582ad80d6 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:42:55.916160 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1137/2294] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:42:55 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:42:56.286755 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce37651-d813-46fd-9d41-eb436495a1ee service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:42:56.295842 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce37651-d813-46fd-9d41-eb436495a1ee service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:42:56.299496 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e4f3f-11c5-443a-879b-909503a52ef3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:42:56.300097 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e4f3f-11c5-443a-879b-909503a52ef3 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:42:56.300383 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e4f3f-11c5-443a-879b-909503a52ef3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:42:56.300669 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e4f3f-11c5-443a-879b-909503a52ef3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:42:56.312489 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e4f3f-11c5-443a-879b-909503a52ef3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:42:56.313974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e4f3f-11c5-443a-879b-909503a52ef3 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:42:56.321859 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8021e5-e48d-4c08-a432-d47582ad80d6 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:42:56.323511 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ee8021e5-e48d-4c08-a432-d47582ad80d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:42:56.326132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1158/2295] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:42:56 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:42:56.338231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8021e5-e48d-4c08-a432-d47582ad80d6 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:42:56.343917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759181a-ebb4-4201-b29d-48e0239c9617 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:42:56.344665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759181a-ebb4-4201-b29d-48e0239c9617 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:42:56.344957 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759181a-ebb4-4201-b29d-48e0239c9617 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:42:56.345279 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759181a-ebb4-4201-b29d-48e0239c9617 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:42:56.353696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759181a-ebb4-4201-b29d-48e0239c9617 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:42:56.355306 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759181a-ebb4-4201-b29d-48e0239c9617 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:42:56.364415 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1138/2296] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:42:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:43:06.320983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=6 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:43:06.360679 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=19 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:43:54.602740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e4f3f-11c5-443a-879b-909503a52ef3 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:43:54.614053 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e4f3f-11c5-443a-879b-909503a52ef3 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:43:54.619822 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af46bfa-fb4f-431e-98db-ca64ddcbf5e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:43:54.620368 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af46bfa-fb4f-431e-98db-ca64ddcbf5e9 service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:43:54.620654 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af46bfa-fb4f-431e-98db-ca64ddcbf5e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:43:54.620931 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af46bfa-fb4f-431e-98db-ca64ddcbf5e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:43:54.628761 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af46bfa-fb4f-431e-98db-ca64ddcbf5e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:43:54.631109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af46bfa-fb4f-431e-98db-ca64ddcbf5e9 service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:43:54.640242 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1159/2297] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:43:54 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:43:54.869229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759181a-ebb4-4201-b29d-48e0239c9617 service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:43:54.880783 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759181a-ebb4-4201-b29d-48e0239c9617 service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:43:54.886604 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee667eaf-b1a6-45ff-87fb-3d8dd21fcec6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:43:54.887214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee667eaf-b1a6-45ff-87fb-3d8dd21fcec6 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:43:54.887498 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee667eaf-b1a6-45ff-87fb-3d8dd21fcec6 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:43:54.887776 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee667eaf-b1a6-45ff-87fb-3d8dd21fcec6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:43:54.896169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee667eaf-b1a6-45ff-87fb-3d8dd21fcec6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:43:54.897771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee667eaf-b1a6-45ff-87fb-3d8dd21fcec6 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:43:54.907702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1139/2298] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:43:54 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:43:55.475081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af46bfa-fb4f-431e-98db-ca64ddcbf5e9 service nova] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:43:55.476767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1af46bfa-fb4f-431e-98db-ca64ddcbf5e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:43:55.492314 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af46bfa-fb4f-431e-98db-ca64ddcbf5e9 service nova] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:43:55.498228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91a281-1dee-4381-beda-da6c9d1cb808 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:43:55.498859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91a281-1dee-4381-beda-da6c9d1cb808 service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:43:55.499195 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91a281-1dee-4381-beda-da6c9d1cb808 service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:43:55.499474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91a281-1dee-4381-beda-da6c9d1cb808 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:43:55.508876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91a281-1dee-4381-beda-da6c9d1cb808 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:43:55.510670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91a281-1dee-4381-beda-da6c9d1cb808 service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:43:55.540650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1160/2299] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:43:55 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:44:00.458816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee667eaf-b1a6-45ff-87fb-3d8dd21fcec6 service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:44:00.468990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee667eaf-b1a6-45ff-87fb-3d8dd21fcec6 service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:44:00.472766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb0a87-c2a4-4bcb-b7ab-42d9f0981b36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:44:00.474268 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb0a87-c2a4-4bcb-b7ab-42d9f0981b36 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:44:00.474780 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb0a87-c2a4-4bcb-b7ab-42d9f0981b36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:44:00.475228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb0a87-c2a4-4bcb-b7ab-42d9f0981b36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:44:00.477033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d2fb0a87-c2a4-4bcb-b7ab-42d9f0981b36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:44:00.575737 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb0a87-c2a4-4bcb-b7ab-42d9f0981b36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:44:00.577226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb0a87-c2a4-4bcb-b7ab-42d9f0981b36 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:44:00.598045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1140/2300] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:44:00 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:44:02.657990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f306f88-8448-459f-98b0-56483bfa3a56 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:44:02.658572 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f306f88-8448-459f-98b0-56483bfa3a56 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:44:02.658693 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f306f88-8448-459f-98b0-56483bfa3a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:44:02.689631 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-8f306f88-8448-459f-98b0-56483bfa3a56 None None] MFA Rules not processed for user `0e93d159e8644203808918c5ae86fc74`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:44:02.690856 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-8f306f88-8448-459f-98b0-56483bfa3a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:44:02.698369 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1161/2301] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:44:02 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:44:03.085634 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb0a87-c2a4-4bcb-b7ab-42d9f0981b36 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:44:03.095250 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb0a87-c2a4-4bcb-b7ab-42d9f0981b36 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:44:03.100820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e219ed8-3a63-416f-b7a2-a124f6227827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:44:03.101562 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e219ed8-3a63-416f-b7a2-a124f6227827 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:44:03.101852 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e219ed8-3a63-416f-b7a2-a124f6227827 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:44:03.102127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e219ed8-3a63-416f-b7a2-a124f6227827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:44:03.103936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1e219ed8-3a63-416f-b7a2-a124f6227827 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:44:03.117050 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e219ed8-3a63-416f-b7a2-a124f6227827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:44:03.118653 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e219ed8-3a63-416f-b7a2-a124f6227827 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:44:03.134377 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1141/2302] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:44:03 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:44:03.477455 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f306f88-8448-459f-98b0-56483bfa3a56 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:44:03.498135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f306f88-8448-459f-98b0-56483bfa3a56 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:44:03.503851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3989c-47c5-4472-83e1-e48e0a7cf9d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b40981fe179a43dc81553d3594dad649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:44:03.505410 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3989c-47c5-4472-83e1-e48e0a7cf9d9 service placement] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:44:03.505715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3989c-47c5-4472-83e1-e48e0a7cf9d9 service placement] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:44:03.505998 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3989c-47c5-4472-83e1-e48e0a7cf9d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:44:03.509061 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-50c3989c-47c5-4472-83e1-e48e0a7cf9d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:44:03.532981 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3989c-47c5-4472-83e1-e48e0a7cf9d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:44:03.534774 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3989c-47c5-4472-83e1-e48e0a7cf9d9 service placement] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:44:03.561328 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1162/2303] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:44:03 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:44:05.929003 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e219ed8-3a63-416f-b7a2-a124f6227827 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:44:05.952031 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e219ed8-3a63-416f-b7a2-a124f6227827 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:44:05.959791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435651f-f4a3-4045-9b38-27da08801125 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b40981fe179a43dc81553d3594dad649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:44:05.964506 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435651f-f4a3-4045-9b38-27da08801125 service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:44:05.966343 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435651f-f4a3-4045-9b38-27da08801125 service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:44:05.966922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435651f-f4a3-4045-9b38-27da08801125 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:44:05.974018 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b435651f-f4a3-4045-9b38-27da08801125 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:44:05.994706 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435651f-f4a3-4045-9b38-27da08801125 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:44:05.996597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435651f-f4a3-4045-9b38-27da08801125 service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:44:06.038646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1142/2304] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:44:05 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:44:06.630230 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3989c-47c5-4472-83e1-e48e0a7cf9d9 service placement] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:44:06.644101 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3989c-47c5-4472-83e1-e48e0a7cf9d9 service placement] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:44:06.651377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc09a58-455c-4714-b09b-90419972636c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:44:06.652563 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc09a58-455c-4714-b09b-90419972636c service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:44:06.653406 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc09a58-455c-4714-b09b-90419972636c service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:44:06.654038 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc09a58-455c-4714-b09b-90419972636c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:44:06.666863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc09a58-455c-4714-b09b-90419972636c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:44:06.669088 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc09a58-455c-4714-b09b-90419972636c service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:44:06.687592 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1163/2305] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:44:06 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:44:10.214175 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435651f-f4a3-4045-9b38-27da08801125 service placement] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:44:10.228163 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435651f-f4a3-4045-9b38-27da08801125 service placement] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:44:10.236337 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048a736-f2ad-422e-937b-fd48da120ae8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:44:10.237372 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048a736-f2ad-422e-937b-fd48da120ae8 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:44:10.237841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048a736-f2ad-422e-937b-fd48da120ae8 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:44:10.238288 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048a736-f2ad-422e-937b-fd48da120ae8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:44:10.240408 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0048a736-f2ad-422e-937b-fd48da120ae8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:44:10.254102 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0048a736-f2ad-422e-937b-fd48da120ae8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:44:10.256631 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0048a736-f2ad-422e-937b-fd48da120ae8 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:44:10.269327 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1143/2306] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:44:10 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:44:10.399225 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc09a58-455c-4714-b09b-90419972636c service nova] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:44:10.412537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc09a58-455c-4714-b09b-90419972636c service nova] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:44:10.418184 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b07855-ee9b-4905-9dd5-4e8156c43547 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b40981fe179a43dc81553d3594dad649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:44:10.420056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b07855-ee9b-4905-9dd5-4e8156c43547 service placement] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:44:10.420607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b07855-ee9b-4905-9dd5-4e8156c43547 service placement] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:44:10.421095 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b07855-ee9b-4905-9dd5-4e8156c43547 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:44:10.423106 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-97b07855-ee9b-4905-9dd5-4e8156c43547 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:44:10.435999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b07855-ee9b-4905-9dd5-4e8156c43547 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:44:10.438052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b07855-ee9b-4905-9dd5-4e8156c43547 service placement] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:44:10.460600 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1164/2307] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:44:10 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:44:10.599647 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048a736-f2ad-422e-937b-fd48da120ae8 service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:44:10.601907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0048a736-f2ad-422e-937b-fd48da120ae8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:44:10.623990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048a736-f2ad-422e-937b-fd48da120ae8 service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:44:10.631731 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006e302-b243-4e1c-910b-63f56f66db9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:44:10.632875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006e302-b243-4e1c-910b-63f56f66db9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:44:10.633366 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006e302-b243-4e1c-910b-63f56f66db9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:44:10.633803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006e302-b243-4e1c-910b-63f56f66db9e admin admin] PATH_INFO: `/v3/users/4f1e5fb6488d4478b8f4449295613e50` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:44:10.635569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006e302-b243-4e1c-910b-63f56f66db9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f1e5fb6488d4478b8f4449295613e50)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:44:10.638449 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006e302-b243-4e1c-910b-63f56f66db9e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:44:10.714860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4006e302-b243-4e1c-910b-63f56f66db9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f1e5fb6488d4478b8f4449295613e50'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:44:10.735816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4006e302-b243-4e1c-910b-63f56f66db9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f1e5fb6488d4478b8f4449295613e50'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:44:10.737507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4006e302-b243-4e1c-910b-63f56f66db9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f1e5fb6488d4478b8f4449295613e50'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:44:10.750040 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4006e302-b243-4e1c-910b-63f56f66db9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f1e5fb6488d4478b8f4449295613e50'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:44:10.752744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4006e302-b243-4e1c-910b-63f56f66db9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f1e5fb6488d4478b8f4449295613e50'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:44:10.761839 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:44:10.788161 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1144/2308] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:44:10 2022] DELETE /identity/v3/users/4f1e5fb6488d4478b8f4449295613e50 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:44:11.071711 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b07855-ee9b-4905-9dd5-4e8156c43547 service placement] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:44:11.095813 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b07855-ee9b-4905-9dd5-4e8156c43547 service placement] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:44:11.238012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea590bca-bc4f-4de9-ba0a-ee9db52389b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:44:11.239765 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea590bca-bc4f-4de9-ba0a-ee9db52389b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:44:11.240374 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea590bca-bc4f-4de9-ba0a-ee9db52389b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:44:11.240903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea590bca-bc4f-4de9-ba0a-ee9db52389b9 admin admin] PATH_INFO: `/v3/projects/a6dd8f336ea44f14bfaf1c127ddbacb1` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:44:11.242730 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea590bca-bc4f-4de9-ba0a-ee9db52389b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6dd8f336ea44f14bfaf1c127ddbacb1)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:44:11.245031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea590bca-bc4f-4de9-ba0a-ee9db52389b9 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:44:11.321706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ea590bca-bc4f-4de9-ba0a-ee9db52389b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6dd8f336ea44f14bfaf1c127ddbacb1', 'request_id': 'req-ea590bca-bc4f-4de9-ba0a-ee9db52389b9'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:44:11.329430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ea590bca-bc4f-4de9-ba0a-ee9db52389b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6dd8f336ea44f14bfaf1c127ddbacb1', 'request_id': 'req-ea590bca-bc4f-4de9-ba0a-ee9db52389b9'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:44:11.340650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ea590bca-bc4f-4de9-ba0a-ee9db52389b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6dd8f336ea44f14bfaf1c127ddbacb1', 'request_id': 'req-ea590bca-bc4f-4de9-ba0a-ee9db52389b9'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:44:11.352705 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:44:11.382878 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ea590bca-bc4f-4de9-ba0a-ee9db52389b9 admin admin] The token cache is being invalidate because project a6dd8f336ea44f14bfaf1c127ddbacb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:44:11.383389 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ea590bca-bc4f-4de9-ba0a-ee9db52389b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:44:11.384350 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ea590bca-bc4f-4de9-ba0a-ee9db52389b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:44:11.389190 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1165/2309] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:44:11 2022] DELETE /identity/v3/projects/a6dd8f336ea44f14bfaf1c127ddbacb1 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:44:11.391102 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006e302-b243-4e1c-910b-63f56f66db9e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:44:11.394138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4006e302-b243-4e1c-910b-63f56f66db9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:44:11.440383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006e302-b243-4e1c-910b-63f56f66db9e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:44:11.653460 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffda79-7b3c-439f-be38-9c86bdfa8227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:44:11.654167 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffda79-7b3c-439f-be38-9c86bdfa8227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:44:11.654458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffda79-7b3c-439f-be38-9c86bdfa8227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:44:11.654884 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffda79-7b3c-439f-be38-9c86bdfa8227 admin admin] PATH_INFO: `/v3/users/26185d043e5d43c79a0f9df21fd2dcca` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:44:11.697981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feffda79-7b3c-439f-be38-9c86bdfa8227 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26185d043e5d43c79a0f9df21fd2dcca)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:44:11.700939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feffda79-7b3c-439f-be38-9c86bdfa8227 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:44:11.773094 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-feffda79-7b3c-439f-be38-9c86bdfa8227 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26185d043e5d43c79a0f9df21fd2dcca'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:44:11.793858 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-feffda79-7b3c-439f-be38-9c86bdfa8227 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26185d043e5d43c79a0f9df21fd2dcca'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:44:11.794696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-feffda79-7b3c-439f-be38-9c86bdfa8227 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26185d043e5d43c79a0f9df21fd2dcca'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:44:11.804718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-feffda79-7b3c-439f-be38-9c86bdfa8227 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26185d043e5d43c79a0f9df21fd2dcca'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:44:11.805547 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-feffda79-7b3c-439f-be38-9c86bdfa8227 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26185d043e5d43c79a0f9df21fd2dcca'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:44:11.829854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1145/2310] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:44:11 2022] DELETE /identity/v3/users/26185d043e5d43c79a0f9df21fd2dcca => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:44:12.043232 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea590bca-bc4f-4de9-ba0a-ee9db52389b9 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:44:12.045003 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ea590bca-bc4f-4de9-ba0a-ee9db52389b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:44:12.074303 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea590bca-bc4f-4de9-ba0a-ee9db52389b9 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:44:12.151592 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0132c-bcab-4f7b-beda-9f6de4e70a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:44:12.152400 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0132c-bcab-4f7b-beda-9f6de4e70a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:44:12.164400 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0132c-bcab-4f7b-beda-9f6de4e70a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:44:12.164956 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0132c-bcab-4f7b-beda-9f6de4e70a6f admin admin] PATH_INFO: `/v3/projects/55ec13d0e96d4465a6df7b3387d8a757` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:44:12.182068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0132c-bcab-4f7b-beda-9f6de4e70a6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=55ec13d0e96d4465a6df7b3387d8a757)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:44:12.184515 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0132c-bcab-4f7b-beda-9f6de4e70a6f admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:44:12.291042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5cc0132c-bcab-4f7b-beda-9f6de4e70a6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55ec13d0e96d4465a6df7b3387d8a757', 'request_id': 'req-5cc0132c-bcab-4f7b-beda-9f6de4e70a6f'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:44:12.311061 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5cc0132c-bcab-4f7b-beda-9f6de4e70a6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55ec13d0e96d4465a6df7b3387d8a757', 'request_id': 'req-5cc0132c-bcab-4f7b-beda-9f6de4e70a6f'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:44:12.326923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5cc0132c-bcab-4f7b-beda-9f6de4e70a6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55ec13d0e96d4465a6df7b3387d8a757', 'request_id': 'req-5cc0132c-bcab-4f7b-beda-9f6de4e70a6f'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:44:12.345351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5cc0132c-bcab-4f7b-beda-9f6de4e70a6f admin admin] The token cache is being invalidate because project 55ec13d0e96d4465a6df7b3387d8a757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:44:12.346669 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5cc0132c-bcab-4f7b-beda-9f6de4e70a6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:44:12.348816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5cc0132c-bcab-4f7b-beda-9f6de4e70a6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:44:12.361175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1166/2311] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:44:12 2022] DELETE /identity/v3/projects/55ec13d0e96d4465a6df7b3387d8a757 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:44:12.682638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffda79-7b3c-439f-be38-9c86bdfa8227 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:44:12.684900 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-feffda79-7b3c-439f-be38-9c86bdfa8227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:44:12.755978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:44:12.785189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffda79-7b3c-439f-be38-9c86bdfa8227 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:44:12.858912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efd9da-4bd2-457e-b814-761178ff8dc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:44:12.859884 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efd9da-4bd2-457e-b814-761178ff8dc3 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:44:12.860168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efd9da-4bd2-457e-b814-761178ff8dc3 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:44:12.860452 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efd9da-4bd2-457e-b814-761178ff8dc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:44:12.862357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-04efd9da-4bd2-457e-b814-761178ff8dc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:44:12.875205 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efd9da-4bd2-457e-b814-761178ff8dc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:44:12.876815 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efd9da-4bd2-457e-b814-761178ff8dc3 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:44:12.889153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1146/2312] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:44:12 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:44:13.579501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caade7a6-d4b1-4bcc-85fa-24760a12835c None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:44:13.580110 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caade7a6-d4b1-4bcc-85fa-24760a12835c None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:44:13.580110 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caade7a6-d4b1-4bcc-85fa-24760a12835c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:44:13.621117 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-caade7a6-d4b1-4bcc-85fa-24760a12835c None None] MFA Rules not processed for user `c9633b9cbd8a4332a605ca66d73dc5b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:44:13.622342 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-caade7a6-d4b1-4bcc-85fa-24760a12835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:44:13.683827 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:44:13.690127 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1167/2313] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:44:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:44:14.062540 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efd9da-4bd2-457e-b814-761178ff8dc3 service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:44:14.063964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-04efd9da-4bd2-457e-b814-761178ff8dc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:44:14.119361 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efd9da-4bd2-457e-b814-761178ff8dc3 service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:44:14.122890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa69730-db86-43f1-985d-20cfee26d05b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:44:14.123498 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa69730-db86-43f1-985d-20cfee26d05b service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:44:14.123778 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa69730-db86-43f1-985d-20cfee26d05b service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:44:14.124049 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa69730-db86-43f1-985d-20cfee26d05b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:44:14.125758 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-efa69730-db86-43f1-985d-20cfee26d05b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:44:14.135190 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa69730-db86-43f1-985d-20cfee26d05b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:44:14.136587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa69730-db86-43f1-985d-20cfee26d05b service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:44:14.144339 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1147/2314] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:44:14 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:44:23.697121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:44:24.140897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=64 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:36.206360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caade7a6-d4b1-4bcc-85fa-24760a12835c None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:36.207833 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-caade7a6-d4b1-4bcc-85fa-24760a12835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:36.282767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caade7a6-d4b1-4bcc-85fa-24760a12835c None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:36.288886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1d347-5054-4643-80ed-4959c15b1e22 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b40981fe179a43dc81553d3594dad649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:36.289538 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1d347-5054-4643-80ed-4959c15b1e22 service placement] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:36.289823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1d347-5054-4643-80ed-4959c15b1e22 service placement] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:36.290100 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1d347-5054-4643-80ed-4959c15b1e22 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:36.291870 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d4d1d347-5054-4643-80ed-4959c15b1e22 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:36.304057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1d347-5054-4643-80ed-4959c15b1e22 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:36.305698 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1d347-5054-4643-80ed-4959c15b1e22 service placement] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:36.316935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1168/2315] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:45:36 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:45:46.311827 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=32 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:52.356761 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa69730-db86-43f1-985d-20cfee26d05b service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:52.370796 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa69730-db86-43f1-985d-20cfee26d05b service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:52.375461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636ab63-411f-4660-b4d2-aa133ed0b565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:52.376168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636ab63-411f-4660-b4d2-aa133ed0b565 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:52.376809 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636ab63-411f-4660-b4d2-aa133ed0b565 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:52.376975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636ab63-411f-4660-b4d2-aa133ed0b565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:52.378806 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6636ab63-411f-4660-b4d2-aa133ed0b565 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:52.456178 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636ab63-411f-4660-b4d2-aa133ed0b565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:52.458241 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636ab63-411f-4660-b4d2-aa133ed0b565 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:52.467506 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1148/2316] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:45:52 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:46:01.216201 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1d347-5054-4643-80ed-4959c15b1e22 service placement] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:01.217847 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d4d1d347-5054-4643-80ed-4959c15b1e22 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:01.232618 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1d347-5054-4643-80ed-4959c15b1e22 service placement] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:01.238582 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22267b-f4b3-4363-b3b1-92f329b7b934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:01.239226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22267b-f4b3-4363-b3b1-92f329b7b934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:01.239514 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22267b-f4b3-4363-b3b1-92f329b7b934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:01.239795 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22267b-f4b3-4363-b3b1-92f329b7b934 admin admin] PATH_INFO: `/v3/users/d22a27db106e45e68444fa022d254c83` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:01.253917 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc22267b-f4b3-4363-b3b1-92f329b7b934 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d22a27db106e45e68444fa022d254c83)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:01.255860 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc22267b-f4b3-4363-b3b1-92f329b7b934 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:01.301933 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-fc22267b-f4b3-4363-b3b1-92f329b7b934 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd22a27db106e45e68444fa022d254c83'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:46:01.302576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-fc22267b-f4b3-4363-b3b1-92f329b7b934 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd22a27db106e45e68444fa022d254c83'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:46:01.309904 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-fc22267b-f4b3-4363-b3b1-92f329b7b934 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd22a27db106e45e68444fa022d254c83'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:46:01.322308 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-fc22267b-f4b3-4363-b3b1-92f329b7b934 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd22a27db106e45e68444fa022d254c83'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:46:01.329463 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-fc22267b-f4b3-4363-b3b1-92f329b7b934 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd22a27db106e45e68444fa022d254c83'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:46:01.335244 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1169/2317] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:46:01 2022] DELETE /identity/v3/users/d22a27db106e45e68444fa022d254c83 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:46:01.523462 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636ab63-411f-4660-b4d2-aa133ed0b565 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:01.541311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636ab63-411f-4660-b4d2-aa133ed0b565 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:01.603716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf83758-9289-4920-9ca2-44953cafb790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:01.604399 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf83758-9289-4920-9ca2-44953cafb790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:01.604686 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf83758-9289-4920-9ca2-44953cafb790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:01.604964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf83758-9289-4920-9ca2-44953cafb790 admin admin] PATH_INFO: `/v3/projects/9a4b022cdf9c448dbb51c5ffd9212181` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:01.616395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf83758-9289-4920-9ca2-44953cafb790 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a4b022cdf9c448dbb51c5ffd9212181)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:01.622038 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf83758-9289-4920-9ca2-44953cafb790 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:01.668008 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7cf83758-9289-4920-9ca2-44953cafb790 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a4b022cdf9c448dbb51c5ffd9212181', 'request_id': 'req-7cf83758-9289-4920-9ca2-44953cafb790'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:46:01.675372 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7cf83758-9289-4920-9ca2-44953cafb790 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a4b022cdf9c448dbb51c5ffd9212181', 'request_id': 'req-7cf83758-9289-4920-9ca2-44953cafb790'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:46:01.682225 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7cf83758-9289-4920-9ca2-44953cafb790 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a4b022cdf9c448dbb51c5ffd9212181', 'request_id': 'req-7cf83758-9289-4920-9ca2-44953cafb790'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:46:01.693410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7cf83758-9289-4920-9ca2-44953cafb790 admin admin] The token cache is being invalidate because project 9a4b022cdf9c448dbb51c5ffd9212181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:46:01.693705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7cf83758-9289-4920-9ca2-44953cafb790 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:46:01.694330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7cf83758-9289-4920-9ca2-44953cafb790 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:46:01.695893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1149/2318] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:46:01 2022] DELETE /identity/v3/projects/9a4b022cdf9c448dbb51c5ffd9212181 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:46:01.700161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22267b-f4b3-4363-b3b1-92f329b7b934 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:01.701583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-fc22267b-f4b3-4363-b3b1-92f329b7b934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:01.714863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22267b-f4b3-4363-b3b1-92f329b7b934 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:01.771436 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c194a-d775-4957-a7a2-f1b216f23a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:01.772077 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c194a-d775-4957-a7a2-f1b216f23a2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:01.772359 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c194a-d775-4957-a7a2-f1b216f23a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:01.772635 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c194a-d775-4957-a7a2-f1b216f23a2f admin admin] PATH_INFO: `/v3/users/6e14cfb4e8fc4c2e8a3f3fd5aa8a7cde` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:01.785790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70c194a-d775-4957-a7a2-f1b216f23a2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e14cfb4e8fc4c2e8a3f3fd5aa8a7cde)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:01.787732 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70c194a-d775-4957-a7a2-f1b216f23a2f admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:01.836284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b70c194a-d775-4957-a7a2-f1b216f23a2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e14cfb4e8fc4c2e8a3f3fd5aa8a7cde'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:46:01.836889 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b70c194a-d775-4957-a7a2-f1b216f23a2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e14cfb4e8fc4c2e8a3f3fd5aa8a7cde'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:46:01.840687 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:01.845736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b70c194a-d775-4957-a7a2-f1b216f23a2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e14cfb4e8fc4c2e8a3f3fd5aa8a7cde'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:46:01.858547 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b70c194a-d775-4957-a7a2-f1b216f23a2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e14cfb4e8fc4c2e8a3f3fd5aa8a7cde'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:46:01.865980 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b70c194a-d775-4957-a7a2-f1b216f23a2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e14cfb4e8fc4c2e8a3f3fd5aa8a7cde'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:46:01.871829 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1170/2319] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:46:01 2022] DELETE /identity/v3/users/6e14cfb4e8fc4c2e8a3f3fd5aa8a7cde => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:46:02.047066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf83758-9289-4920-9ca2-44953cafb790 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:02.049241 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7cf83758-9289-4920-9ca2-44953cafb790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:02.062407 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf83758-9289-4920-9ca2-44953cafb790 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:02.080405 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:02.133969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1232e-531a-41fb-aac9-c160cfc51584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:02.134711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1232e-531a-41fb-aac9-c160cfc51584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:02.135060 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1232e-531a-41fb-aac9-c160cfc51584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:02.135564 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1232e-531a-41fb-aac9-c160cfc51584 admin admin] PATH_INFO: `/v3/projects/29849de6f58e47e7b7ba96234cbfe1b2` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:02.157240 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1232e-531a-41fb-aac9-c160cfc51584 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29849de6f58e47e7b7ba96234cbfe1b2)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:02.159346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1232e-531a-41fb-aac9-c160cfc51584 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:02.208768 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d0f1232e-531a-41fb-aac9-c160cfc51584 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29849de6f58e47e7b7ba96234cbfe1b2', 'request_id': 'req-d0f1232e-531a-41fb-aac9-c160cfc51584'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:46:02.215672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d0f1232e-531a-41fb-aac9-c160cfc51584 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29849de6f58e47e7b7ba96234cbfe1b2', 'request_id': 'req-d0f1232e-531a-41fb-aac9-c160cfc51584'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:46:02.226318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d0f1232e-531a-41fb-aac9-c160cfc51584 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29849de6f58e47e7b7ba96234cbfe1b2', 'request_id': 'req-d0f1232e-531a-41fb-aac9-c160cfc51584'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:46:02.237986 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d0f1232e-531a-41fb-aac9-c160cfc51584 admin admin] The token cache is being invalidate because project 29849de6f58e47e7b7ba96234cbfe1b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:46:02.238329 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d0f1232e-531a-41fb-aac9-c160cfc51584 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:46:02.238986 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d0f1232e-531a-41fb-aac9-c160cfc51584 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:46:02.240545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1150/2320] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:46:02 2022] DELETE /identity/v3/projects/29849de6f58e47e7b7ba96234cbfe1b2 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:46:08.042034 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e1431-1756-4efa-908e-da9252b9e7e4 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:08.042500 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e1431-1756-4efa-908e-da9252b9e7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:08.042610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e1431-1756-4efa-908e-da9252b9e7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:08.061420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-686e1431-1756-4efa-908e-da9252b9e7e4 None None] MFA Rules not processed for user `c9633b9cbd8a4332a605ca66d73dc5b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:08.063200 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-686e1431-1756-4efa-908e-da9252b9e7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:08.118995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1171/2321] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:46:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:46:10.532966 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf322b7-71b6-42e5-8ef8-b72e9ae8ca16 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:10.533804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf322b7-71b6-42e5-8ef8-b72e9ae8ca16 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:10.534427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf322b7-71b6-42e5-8ef8-b72e9ae8ca16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:10.608719 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-eaf322b7-71b6-42e5-8ef8-b72e9ae8ca16 None None] MFA Rules not processed for user `1cd36b543bd9487ebd8ce578e9a76c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:10.613348 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-eaf322b7-71b6-42e5-8ef8-b72e9ae8ca16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:10.678074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:10.685407 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1151/2322] 10.222.0.100 () {58 vars in 1014 bytes} [Tue Aug 2 09:46:10 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:46:10.734217 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e1431-1756-4efa-908e-da9252b9e7e4 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:10.736301 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-686e1431-1756-4efa-908e-da9252b9e7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:10.857571 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e1431-1756-4efa-908e-da9252b9e7e4 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:10.863814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b910b2-f70b-4a2d-86a8-d474a5e14f17 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50cb532b58ed4951ba960a0a7d579b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:10.864372 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b910b2-f70b-4a2d-86a8-d474a5e14f17 service swift] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:10.864659 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b910b2-f70b-4a2d-86a8-d474a5e14f17 service swift] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:10.865036 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b910b2-f70b-4a2d-86a8-d474a5e14f17 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:10.876399 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b910b2-f70b-4a2d-86a8-d474a5e14f17 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:10.878084 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b910b2-f70b-4a2d-86a8-d474a5e14f17 service swift] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:10.888413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1172/2323] 10.222.0.100 () {60 vars in 1389 bytes} [Tue Aug 2 09:46:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:46:11.186066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f829d-7b3b-4fe2-bea6-17a86c3b5701 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:11.186582 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f829d-7b3b-4fe2-bea6-17a86c3b5701 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:11.186870 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f829d-7b3b-4fe2-bea6-17a86c3b5701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:11.242282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a79f829d-7b3b-4fe2-bea6-17a86c3b5701 None None] MFA Rules not processed for user `b074a126a458485e882a9e9eea178dca`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:11.244478 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a79f829d-7b3b-4fe2-bea6-17a86c3b5701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:11.350627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1152/2324] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:46:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:46:20.894018 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=78,keystone:DELETE=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:21.322398 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=28 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:27.191469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176605d7-d388-4ab6-bb81-322f41d9c351 None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:27.192042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176605d7-d388-4ab6-bb81-322f41d9c351 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:27.192151 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176605d7-d388-4ab6-bb81-322f41d9c351 None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:27.193514 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1173/2325] 10.222.0.100 () {54 vars in 926 bytes} [Tue Aug 2 09:46:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:46:27.200138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf563b-908e-44f8-b3d5-b69b210e6ffc None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:27.200624 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf563b-908e-44f8-b3d5-b69b210e6ffc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:27.200740 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf563b-908e-44f8-b3d5-b69b210e6ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:27.252603 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-01bf563b-908e-44f8-b3d5-b69b210e6ffc None None] MFA Rules not processed for user `49a0c1a4a6cf43e28ec1b8488c681848`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:27.254079 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-01bf563b-908e-44f8-b3d5-b69b210e6ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:27.326154 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1153/2326] 10.222.0.100 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:27 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:46:27.339611 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176605d7-d388-4ab6-bb81-322f41d9c351 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:27.341379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-176605d7-d388-4ab6-bb81-322f41d9c351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:27.423753 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176605d7-d388-4ab6-bb81-322f41d9c351 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:27.428627 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7889eb-acaa-4a16-85ca-047af5cb2b44 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd959ddd087724547a11c3d4c796d8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:27.430192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7889eb-acaa-4a16-85ca-047af5cb2b44 service glance] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:27.430659 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7889eb-acaa-4a16-85ca-047af5cb2b44 service glance] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:27.431110 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7889eb-acaa-4a16-85ca-047af5cb2b44 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:27.446649 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7889eb-acaa-4a16-85ca-047af5cb2b44 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:27.448140 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7889eb-acaa-4a16-85ca-047af5cb2b44 service glance] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:27.459316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1174/2327] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:46:27 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:46:27.528555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed211c3-18b0-4de3-ac1e-a8a14b85ac62 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:27.528827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed211c3-18b0-4de3-ac1e-a8a14b85ac62 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:27.529137 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed211c3-18b0-4de3-ac1e-a8a14b85ac62 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:27.530514 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1154/2328] 10.222.0.100 () {54 vars in 926 bytes} [Tue Aug 2 09:46:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:46:27.535826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bbfc5-4167-47a6-a59a-5a4ba4276c60 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:27.536184 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bbfc5-4167-47a6-a59a-5a4ba4276c60 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:27.536528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bbfc5-4167-47a6-a59a-5a4ba4276c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:27.577585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-3a8bbfc5-4167-47a6-a59a-5a4ba4276c60 None None] MFA Rules not processed for user `f4afef33deb247509765a8c88a78ac39`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:27.579431 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3a8bbfc5-4167-47a6-a59a-5a4ba4276c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:27.643113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1175/2329] 10.222.0.100 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:27 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:46:27.655492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed211c3-18b0-4de3-ac1e-a8a14b85ac62 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:27.658068 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1ed211c3-18b0-4de3-ac1e-a8a14b85ac62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:27.671232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed211c3-18b0-4de3-ac1e-a8a14b85ac62 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:27.675661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501107ee-b24f-43b1-a256-986e2c1fb32b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:27.676275 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501107ee-b24f-43b1-a256-986e2c1fb32b service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:27.676559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501107ee-b24f-43b1-a256-986e2c1fb32b service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:27.676839 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501107ee-b24f-43b1-a256-986e2c1fb32b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:27.684366 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501107ee-b24f-43b1-a256-986e2c1fb32b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:27.685849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501107ee-b24f-43b1-a256-986e2c1fb32b service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:27.694073 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1155/2330] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:46:27 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:46:27.781874 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f548b81-99f5-4386-a3ca-bcd9f63a2abf None None] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:27.782172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f548b81-99f5-4386-a3ca-bcd9f63a2abf None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:27.782459 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f548b81-99f5-4386-a3ca-bcd9f63a2abf None None] PATH_INFO: `/` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:27.783846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1176/2331] 10.222.0.100 () {54 vars in 926 bytes} [Tue Aug 2 09:46:27 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 09:46:27.789199 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f2355-f061-47e5-938e-f9b0431175ff None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:27.789492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f2355-f061-47e5-938e-f9b0431175ff None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:27.789773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f2355-f061-47e5-938e-f9b0431175ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:27.809847 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-433f2355-f061-47e5-938e-f9b0431175ff None None] MFA Rules not processed for user `49a0c1a4a6cf43e28ec1b8488c681848`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:27.811239 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-433f2355-f061-47e5-938e-f9b0431175ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:27.822554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1156/2332] 10.222.0.100 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:27 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:46:27.834152 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f548b81-99f5-4386-a3ca-bcd9f63a2abf None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:27.846494 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f548b81-99f5-4386-a3ca-bcd9f63a2abf None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:27.850108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c5e31-891d-42b2-8b64-d2e9c4697ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:27.850740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c5e31-891d-42b2-8b64-d2e9c4697ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:27.851027 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c5e31-891d-42b2-8b64-d2e9c4697ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:27.851304 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c5e31-891d-42b2-8b64-d2e9c4697ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:27.864098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c5e31-891d-42b2-8b64-d2e9c4697ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:27.865584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c5e31-891d-42b2-8b64-d2e9c4697ef4 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:27.877009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1177/2333] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:46:27 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:46:29.639243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f70a0a-0489-48d7-ae29-fc9ab5be0302 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:29.639728 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f70a0a-0489-48d7-ae29-fc9ab5be0302 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:29.639827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f70a0a-0489-48d7-ae29-fc9ab5be0302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:29.660434 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d2f70a0a-0489-48d7-ae29-fc9ab5be0302 None None] MFA Rules not processed for user `1cd36b543bd9487ebd8ce578e9a76c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:29.661787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d2f70a0a-0489-48d7-ae29-fc9ab5be0302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:29.672706 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1157/2334] 10.222.0.100 () {58 vars in 1014 bytes} [Tue Aug 2 09:46:29 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:46:29.685335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c5e31-891d-42b2-8b64-d2e9c4697ef4 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:29.697278 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c5e31-891d-42b2-8b64-d2e9c4697ef4 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:29.702853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a298f9-8931-49f6-8dcf-a86d6e3fbad5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50cb532b58ed4951ba960a0a7d579b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:29.703541 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a298f9-8931-49f6-8dcf-a86d6e3fbad5 service swift] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:29.703935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a298f9-8931-49f6-8dcf-a86d6e3fbad5 service swift] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:29.704431 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a298f9-8931-49f6-8dcf-a86d6e3fbad5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:29.722208 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a298f9-8931-49f6-8dcf-a86d6e3fbad5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:29.724276 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a298f9-8931-49f6-8dcf-a86d6e3fbad5 service swift] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:29.733560 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1178/2335] 10.222.0.100 () {60 vars in 1389 bytes} [Tue Aug 2 09:46:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:46:39.654338 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=53 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:39.730143 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=76 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:51.991617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8473e49-fd58-4b66-a94a-2eb611a55f5a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:51.992156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8473e49-fd58-4b66-a94a-2eb611a55f5a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:51.992156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8473e49-fd58-4b66-a94a-2eb611a55f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:52.013441 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c8473e49-fd58-4b66-a94a-2eb611a55f5a None None] MFA Rules not processed for user `1cd36b543bd9487ebd8ce578e9a76c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:52.015379 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c8473e49-fd58-4b66-a94a-2eb611a55f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:52.022626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1158/2336] 10.222.0.100 () {58 vars in 972 bytes} [Tue Aug 2 09:46:51 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:46:52.035686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a298f9-8931-49f6-8dcf-a86d6e3fbad5 service swift] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:52.046609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a298f9-8931-49f6-8dcf-a86d6e3fbad5 service swift] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:52.051936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c868d-8160-4656-af27-ff88a5139534 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50cb532b58ed4951ba960a0a7d579b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:52.052559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c868d-8160-4656-af27-ff88a5139534 service swift] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:52.052846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c868d-8160-4656-af27-ff88a5139534 service swift] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:52.053150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c868d-8160-4656-af27-ff88a5139534 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:52.062425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c868d-8160-4656-af27-ff88a5139534 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:52.064010 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c868d-8160-4656-af27-ff88a5139534 service swift] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:52.072304 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1179/2337] 10.222.0.100 () {60 vars in 1389 bytes} [Tue Aug 2 09:46:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:46:57.432783 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8473e49-fd58-4b66-a94a-2eb611a55f5a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:57.434290 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c8473e49-fd58-4b66-a94a-2eb611a55f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:57.506405 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8473e49-fd58-4b66-a94a-2eb611a55f5a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:57.510304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee9053-1f47-4012-ae35-4aef64885935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:57.510939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee9053-1f47-4012-ae35-4aef64885935 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:57.511220 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee9053-1f47-4012-ae35-4aef64885935 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:57.511494 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee9053-1f47-4012-ae35-4aef64885935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:57.513417 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-68ee9053-1f47-4012-ae35-4aef64885935 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:57.591359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee9053-1f47-4012-ae35-4aef64885935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:57.593526 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee9053-1f47-4012-ae35-4aef64885935 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:57.622053 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1159/2338] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:46:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:47:02.069199 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=6 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:07.619088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=64 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:26.867641 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c868d-8160-4656-af27-ff88a5139534 service swift] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:26.869897 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-817c868d-8160-4656-af27-ff88a5139534 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:26.897642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c868d-8160-4656-af27-ff88a5139534 service swift] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:26.901336 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e15eb8-cad7-4ac1-b18b-8fbfa09f9328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:26.901967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e15eb8-cad7-4ac1-b18b-8fbfa09f9328 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:26.902255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e15eb8-cad7-4ac1-b18b-8fbfa09f9328 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:26.902536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e15eb8-cad7-4ac1-b18b-8fbfa09f9328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:26.904723 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c6e15eb8-cad7-4ac1-b18b-8fbfa09f9328 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:26.921163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e15eb8-cad7-4ac1-b18b-8fbfa09f9328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:26.922582 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e15eb8-cad7-4ac1-b18b-8fbfa09f9328 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:26.931258 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1180/2339] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:47:26 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:47:27.556214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee9053-1f47-4012-ae35-4aef64885935 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:27.566213 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee9053-1f47-4012-ae35-4aef64885935 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:27.570091 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689c122-a297-4b9d-af16-2fcdcd730226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:27.570747 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689c122-a297-4b9d-af16-2fcdcd730226 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:27.571032 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689c122-a297-4b9d-af16-2fcdcd730226 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:27.571310 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689c122-a297-4b9d-af16-2fcdcd730226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:27.573213 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5689c122-a297-4b9d-af16-2fcdcd730226 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:27.647347 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5689c122-a297-4b9d-af16-2fcdcd730226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:27.648791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5689c122-a297-4b9d-af16-2fcdcd730226 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:27.660146 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1160/2340] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:47:27 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:47:28.216100 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e15eb8-cad7-4ac1-b18b-8fbfa09f9328 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:28.217961 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c6e15eb8-cad7-4ac1-b18b-8fbfa09f9328 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:28.242032 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e15eb8-cad7-4ac1-b18b-8fbfa09f9328 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:28.246571 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594916fe-fe18-4cc1-9976-ecfd1ce46fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:28.248141 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594916fe-fe18-4cc1-9976-ecfd1ce46fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:28.248559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594916fe-fe18-4cc1-9976-ecfd1ce46fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:28.248897 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594916fe-fe18-4cc1-9976-ecfd1ce46fe2 admin admin] PATH_INFO: `/v3/users/b074a126a458485e882a9e9eea178dca` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:28.250219 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594916fe-fe18-4cc1-9976-ecfd1ce46fe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b074a126a458485e882a9e9eea178dca)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:28.252229 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594916fe-fe18-4cc1-9976-ecfd1ce46fe2 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:28.302949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-594916fe-fe18-4cc1-9976-ecfd1ce46fe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b074a126a458485e882a9e9eea178dca'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:28.303686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-594916fe-fe18-4cc1-9976-ecfd1ce46fe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b074a126a458485e882a9e9eea178dca'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:28.312182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-594916fe-fe18-4cc1-9976-ecfd1ce46fe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b074a126a458485e882a9e9eea178dca'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:28.326040 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-594916fe-fe18-4cc1-9976-ecfd1ce46fe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b074a126a458485e882a9e9eea178dca'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:28.333519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-594916fe-fe18-4cc1-9976-ecfd1ce46fe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b074a126a458485e882a9e9eea178dca'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:28.339763 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1181/2341] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:47:28 2022] DELETE /identity/v3/users/b074a126a458485e882a9e9eea178dca => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:28.519428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689c122-a297-4b9d-af16-2fcdcd730226 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:28.530042 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689c122-a297-4b9d-af16-2fcdcd730226 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:28.590944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf33984-6c00-49d1-98a1-eceb3d94ea2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:28.591635 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf33984-6c00-49d1-98a1-eceb3d94ea2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:28.591863 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf33984-6c00-49d1-98a1-eceb3d94ea2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:28.592138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf33984-6c00-49d1-98a1-eceb3d94ea2a admin admin] PATH_INFO: `/v3/projects/0d0ba64c011a419b9cc8f5501a8606ac` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:28.593393 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf33984-6c00-49d1-98a1-eceb3d94ea2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d0ba64c011a419b9cc8f5501a8606ac)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:28.595353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf33984-6c00-49d1-98a1-eceb3d94ea2a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:28.644175 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ebf33984-6c00-49d1-98a1-eceb3d94ea2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d0ba64c011a419b9cc8f5501a8606ac', 'request_id': 'req-ebf33984-6c00-49d1-98a1-eceb3d94ea2a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:28.650261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ebf33984-6c00-49d1-98a1-eceb3d94ea2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d0ba64c011a419b9cc8f5501a8606ac', 'request_id': 'req-ebf33984-6c00-49d1-98a1-eceb3d94ea2a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:28.659668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ebf33984-6c00-49d1-98a1-eceb3d94ea2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d0ba64c011a419b9cc8f5501a8606ac', 'request_id': 'req-ebf33984-6c00-49d1-98a1-eceb3d94ea2a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:28.670780 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ebf33984-6c00-49d1-98a1-eceb3d94ea2a admin admin] The token cache is being invalidate because project 0d0ba64c011a419b9cc8f5501a8606ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:47:28.671078 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ebf33984-6c00-49d1-98a1-eceb3d94ea2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:28.671752 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ebf33984-6c00-49d1-98a1-eceb3d94ea2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:28.673419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1161/2342] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:47:28 2022] DELETE /identity/v3/projects/0d0ba64c011a419b9cc8f5501a8606ac => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:31.922662 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594916fe-fe18-4cc1-9976-ecfd1ce46fe2 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:31.923898 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-594916fe-fe18-4cc1-9976-ecfd1ce46fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:31.985029 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594916fe-fe18-4cc1-9976-ecfd1ce46fe2 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:32.036160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4e1f2-534a-488f-91f6-2cf0757976b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:32.036781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4e1f2-534a-488f-91f6-2cf0757976b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:32.037066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4e1f2-534a-488f-91f6-2cf0757976b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:32.037376 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4e1f2-534a-488f-91f6-2cf0757976b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:32.039159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-92b4e1f2-534a-488f-91f6-2cf0757976b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:32.057819 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:32.105383 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4e1f2-534a-488f-91f6-2cf0757976b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:32.107468 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4e1f2-534a-488f-91f6-2cf0757976b4 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:32.117204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1182/2343] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:47:31 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:47:35.413397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf33984-6c00-49d1-98a1-eceb3d94ea2a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:35.415054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ebf33984-6c00-49d1-98a1-eceb3d94ea2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:35.430627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf33984-6c00-49d1-98a1-eceb3d94ea2a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:35.435681 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb232a-8915-4139-88d2-1a7c56cce571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:35.436639 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb232a-8915-4139-88d2-1a7c56cce571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:35.436990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb232a-8915-4139-88d2-1a7c56cce571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:35.437325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb232a-8915-4139-88d2-1a7c56cce571 admin admin] PATH_INFO: `/v3/users/c9633b9cbd8a4332a605ca66d73dc5b1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:35.438524 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb232a-8915-4139-88d2-1a7c56cce571 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9633b9cbd8a4332a605ca66d73dc5b1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:35.440463 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb232a-8915-4139-88d2-1a7c56cce571 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:35.463578 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:35.498170 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-94bb232a-8915-4139-88d2-1a7c56cce571 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9633b9cbd8a4332a605ca66d73dc5b1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:35.505202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-94bb232a-8915-4139-88d2-1a7c56cce571 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9633b9cbd8a4332a605ca66d73dc5b1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:35.506173 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-94bb232a-8915-4139-88d2-1a7c56cce571 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9633b9cbd8a4332a605ca66d73dc5b1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:35.513801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-94bb232a-8915-4139-88d2-1a7c56cce571 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9633b9cbd8a4332a605ca66d73dc5b1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:35.514462 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-94bb232a-8915-4139-88d2-1a7c56cce571 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9633b9cbd8a4332a605ca66d73dc5b1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:35.532453 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1162/2344] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:47:35 2022] DELETE /identity/v3/users/c9633b9cbd8a4332a605ca66d73dc5b1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:35.709321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4e1f2-534a-488f-91f6-2cf0757976b4 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:35.720853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4e1f2-534a-488f-91f6-2cf0757976b4 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:35.779011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816f2b6-20c6-41e0-b4b5-ff06a0442a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:35.779680 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816f2b6-20c6-41e0-b4b5-ff06a0442a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:35.780068 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816f2b6-20c6-41e0-b4b5-ff06a0442a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:35.780365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816f2b6-20c6-41e0-b4b5-ff06a0442a3c admin admin] PATH_INFO: `/v3/projects/05905727ed6c41a1b6187fec6e61dd8f` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:35.781577 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816f2b6-20c6-41e0-b4b5-ff06a0442a3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=05905727ed6c41a1b6187fec6e61dd8f)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:35.783497 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816f2b6-20c6-41e0-b4b5-ff06a0442a3c admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:35.831647 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-7816f2b6-20c6-41e0-b4b5-ff06a0442a3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05905727ed6c41a1b6187fec6e61dd8f', 'request_id': 'req-7816f2b6-20c6-41e0-b4b5-ff06a0442a3c'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:35.837847 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-7816f2b6-20c6-41e0-b4b5-ff06a0442a3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05905727ed6c41a1b6187fec6e61dd8f', 'request_id': 'req-7816f2b6-20c6-41e0-b4b5-ff06a0442a3c'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:35.845977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-7816f2b6-20c6-41e0-b4b5-ff06a0442a3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05905727ed6c41a1b6187fec6e61dd8f', 'request_id': 'req-7816f2b6-20c6-41e0-b4b5-ff06a0442a3c'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:35.856511 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-7816f2b6-20c6-41e0-b4b5-ff06a0442a3c admin admin] The token cache is being invalidate because project 05905727ed6c41a1b6187fec6e61dd8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:47:35.856799 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-7816f2b6-20c6-41e0-b4b5-ff06a0442a3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:35.857450 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-7816f2b6-20c6-41e0-b4b5-ff06a0442a3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:35.858965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1183/2345] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:47:35 2022] DELETE /identity/v3/projects/05905727ed6c41a1b6187fec6e61dd8f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:35.864042 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff67e2-8f9e-48cf-8353-50aa284e6818 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:35.864342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff67e2-8f9e-48cf-8353-50aa284e6818 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:35.864626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff67e2-8f9e-48cf-8353-50aa284e6818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:35.916692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-14ff67e2-8f9e-48cf-8353-50aa284e6818 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:35.917941 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-14ff67e2-8f9e-48cf-8353-50aa284e6818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:35.979314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1163/2346] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:47:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:47:35.987336 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74983ea-af52-4237-8b2a-27df81705bc3 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:35.987698 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74983ea-af52-4237-8b2a-27df81705bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:35.988042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74983ea-af52-4237-8b2a-27df81705bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:36.008605 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-e74983ea-af52-4237-8b2a-27df81705bc3 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:36.009852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e74983ea-af52-4237-8b2a-27df81705bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:36.018513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1184/2347] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:47:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:47:36.022953 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff67e2-8f9e-48cf-8353-50aa284e6818 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:36.032900 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff67e2-8f9e-48cf-8353-50aa284e6818 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:36.037935 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0a222-2382-4938-a020-5c470062bd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:36.038575 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0a222-2382-4938-a020-5c470062bd29 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:36.038862 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0a222-2382-4938-a020-5c470062bd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:36.039145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0a222-2382-4938-a020-5c470062bd29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:36.039841 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0a222-2382-4938-a020-5c470062bd29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:36.040146 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0a222-2382-4938-a020-5c470062bd29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:36.041789 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0a222-2382-4938-a020-5c470062bd29 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:36.053923 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1164/2348] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:47:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:36.058128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74983ea-af52-4237-8b2a-27df81705bc3 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:36.067781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74983ea-af52-4237-8b2a-27df81705bc3 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:36.072362 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa90c9-c35b-40bd-b99c-e926a09c5591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:36.072959 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa90c9-c35b-40bd-b99c-e926a09c5591 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:36.073270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa90c9-c35b-40bd-b99c-e926a09c5591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:36.073548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa90c9-c35b-40bd-b99c-e926a09c5591 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:36.074199 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa90c9-c35b-40bd-b99c-e926a09c5591 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:36.076875 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa90c9-c35b-40bd-b99c-e926a09c5591 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:36.114695 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1185/2349] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:47:36 2022] POST /identity/v3/projects => generated 386 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:36.118970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0a222-2382-4938-a020-5c470062bd29 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:36.130185 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0a222-2382-4938-a020-5c470062bd29 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:36.187900 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b137d1b-fa55-4831-bde1-3e5e7b54ad7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:36.188532 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b137d1b-fa55-4831-bde1-3e5e7b54ad7b admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:36.188828 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b137d1b-fa55-4831-bde1-3e5e7b54ad7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:36.189192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b137d1b-fa55-4831-bde1-3e5e7b54ad7b admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:36.189823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b137d1b-fa55-4831-bde1-3e5e7b54ad7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:36.191733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b137d1b-fa55-4831-bde1-3e5e7b54ad7b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:36.201449 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:36.216886 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1165/2350] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:47:36 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:47:36.220958 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa90c9-c35b-40bd-b99c-e926a09c5591 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:36.231377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa90c9-c35b-40bd-b99c-e926a09c5591 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:36.235954 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8311c4a-756c-4169-ab49-0e8d5926eb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:36.236561 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8311c4a-756c-4169-ab49-0e8d5926eb3f admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:36.236843 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8311c4a-756c-4169-ab49-0e8d5926eb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:36.237139 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8311c4a-756c-4169-ab49-0e8d5926eb3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:36.237830 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8311c4a-756c-4169-ab49-0e8d5926eb3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:36.239423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8311c4a-756c-4169-ab49-0e8d5926eb3f admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:36.240710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:36.251618 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1186/2351] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:47:36 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:47:36.255655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b137d1b-fa55-4831-bde1-3e5e7b54ad7b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:36.267107 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b137d1b-fa55-4831-bde1-3e5e7b54ad7b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:36.271799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad17af5-205e-4ff1-9326-03e02a389f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:36.272412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad17af5-205e-4ff1-9326-03e02a389f60 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:36.272700 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad17af5-205e-4ff1-9326-03e02a389f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:36.272982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad17af5-205e-4ff1-9326-03e02a389f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:36.273735 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad17af5-205e-4ff1-9326-03e02a389f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:36.275331 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad17af5-205e-4ff1-9326-03e02a389f60 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:36.286679 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1166/2352] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:47:36 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:47:36.290858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8311c4a-756c-4169-ab49-0e8d5926eb3f admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:36.300940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8311c4a-756c-4169-ab49-0e8d5926eb3f admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:36.305562 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ee5fb-9eed-403c-8d17-c1e90cb54429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:36.306211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ee5fb-9eed-403c-8d17-c1e90cb54429 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:36.306495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ee5fb-9eed-403c-8d17-c1e90cb54429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:36.306772 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ee5fb-9eed-403c-8d17-c1e90cb54429 admin admin] PATH_INFO: `/v3/projects/0baa696bf0e14062b974d2eac6edb520/users/715b6c0ca114451dba6aa6c59b6b3ee7/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:36.321897 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ee5fb-9eed-403c-8d17-c1e90cb54429 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0baa696bf0e14062b974d2eac6edb520, user_id=715b6c0ca114451dba6aa6c59b6b3ee7, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:36.325104 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ee5fb-9eed-403c-8d17-c1e90cb54429 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:36.338902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1187/2353] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:47:36 2022] PUT /identity/v3/projects/0baa696bf0e14062b974d2eac6edb520/users/715b6c0ca114451dba6aa6c59b6b3ee7/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:47:36.348802 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e2d19-83d3-4d2b-b82b-7540fed802fe None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:36.349135 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e2d19-83d3-4d2b-b82b-7540fed802fe None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:36.349430 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e2d19-83d3-4d2b-b82b-7540fed802fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:36.378465 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f53e2d19-83d3-4d2b-b82b-7540fed802fe None None] MFA Rules not processed for user `715b6c0ca114451dba6aa6c59b6b3ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:36.379678 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f53e2d19-83d3-4d2b-b82b-7540fed802fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:36.446145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1167/2354] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:47:36 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:47:36.457959 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ee5fb-9eed-403c-8d17-c1e90cb54429 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:36.459711 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-148ee5fb-9eed-403c-8d17-c1e90cb54429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:36.537537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ee5fb-9eed-403c-8d17-c1e90cb54429 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:36.543427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c7036-3fde-46ff-aa74-3f83492dbfd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:36.544137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c7036-3fde-46ff-aa74-3f83492dbfd5 service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:36.544420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c7036-3fde-46ff-aa74-3f83492dbfd5 service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:36.544696 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c7036-3fde-46ff-aa74-3f83492dbfd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:36.552995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c7036-3fde-46ff-aa74-3f83492dbfd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:36.554955 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c7036-3fde-46ff-aa74-3f83492dbfd5 service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:36.564528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1188/2355] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:47:36 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:47:44.008567 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e2d19-83d3-4d2b-b82b-7540fed802fe None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:44.010302 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f53e2d19-83d3-4d2b-b82b-7540fed802fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:44.070696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e2d19-83d3-4d2b-b82b-7540fed802fe None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:44.074453 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31271f50-3487-4b71-9655-f52bb2c9310a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:44.075087 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31271f50-3487-4b71-9655-f52bb2c9310a service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:44.075365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31271f50-3487-4b71-9655-f52bb2c9310a service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:44.075758 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31271f50-3487-4b71-9655-f52bb2c9310a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:44.077613 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-31271f50-3487-4b71-9655-f52bb2c9310a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:44.149062 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31271f50-3487-4b71-9655-f52bb2c9310a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:44.150544 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31271f50-3487-4b71-9655-f52bb2c9310a service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:44.160861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1168/2356] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:47:44 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:47:44.999994 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c7036-3fde-46ff-aa74-3f83492dbfd5 service nova] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:45.011014 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c7036-3fde-46ff-aa74-3f83492dbfd5 service nova] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:45.016255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ea428-08b4-499b-a51a-f94bfe7497b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:45.016895 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ea428-08b4-499b-a51a-f94bfe7497b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:45.017215 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ea428-08b4-499b-a51a-f94bfe7497b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:45.017493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ea428-08b4-499b-a51a-f94bfe7497b9 admin admin] PATH_INFO: `/v3/users/715b6c0ca114451dba6aa6c59b6b3ee7` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:45.018660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ea428-08b4-499b-a51a-f94bfe7497b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=715b6c0ca114451dba6aa6c59b6b3ee7)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:45.020579 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ea428-08b4-499b-a51a-f94bfe7497b9 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:45.076067 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dd0ea428-08b4-499b-a51a-f94bfe7497b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '715b6c0ca114451dba6aa6c59b6b3ee7'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:45.076767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dd0ea428-08b4-499b-a51a-f94bfe7497b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '715b6c0ca114451dba6aa6c59b6b3ee7'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:45.086115 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dd0ea428-08b4-499b-a51a-f94bfe7497b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '715b6c0ca114451dba6aa6c59b6b3ee7'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:45.101940 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dd0ea428-08b4-499b-a51a-f94bfe7497b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '715b6c0ca114451dba6aa6c59b6b3ee7'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:45.111016 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dd0ea428-08b4-499b-a51a-f94bfe7497b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '715b6c0ca114451dba6aa6c59b6b3ee7'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:45.117629 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1189/2357] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:47:44 2022] DELETE /identity/v3/users/715b6c0ca114451dba6aa6c59b6b3ee7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:45.123133 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdef596-0ef9-4b28-90a5-80d7531671db None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:45.123659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdef596-0ef9-4b28-90a5-80d7531671db None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:45.123869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdef596-0ef9-4b28-90a5-80d7531671db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:45.145343 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-4fdef596-0ef9-4b28-90a5-80d7531671db None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:45.146958 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4fdef596-0ef9-4b28-90a5-80d7531671db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:45.166083 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:45.231696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1169/2358] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:47:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:47:45.242096 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ea428-08b4-499b-a51a-f94bfe7497b9 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:45.243713 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-dd0ea428-08b4-499b-a51a-f94bfe7497b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:45.281352 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:45.310352 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ea428-08b4-499b-a51a-f94bfe7497b9 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:45.314149 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d2673-2e6e-4cd1-b57e-e6bcffb0ce3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:45.314778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d2673-2e6e-4cd1-b57e-e6bcffb0ce3c service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:45.315063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d2673-2e6e-4cd1-b57e-e6bcffb0ce3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:45.315341 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d2673-2e6e-4cd1-b57e-e6bcffb0ce3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:45.325780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d2673-2e6e-4cd1-b57e-e6bcffb0ce3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:45.327233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d2673-2e6e-4cd1-b57e-e6bcffb0ce3c service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:45.337005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1190/2359] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:47:45 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:47:45.641855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdef596-0ef9-4b28-90a5-80d7531671db None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:45.645906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4fdef596-0ef9-4b28-90a5-80d7531671db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:45.677362 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdef596-0ef9-4b28-90a5-80d7531671db None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:45.683960 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc55c9-cbbe-4f39-acf2-177e7545c1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:45.685011 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc55c9-cbbe-4f39-acf2-177e7545c1bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:45.685550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc55c9-cbbe-4f39-acf2-177e7545c1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:45.686117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc55c9-cbbe-4f39-acf2-177e7545c1bf admin admin] PATH_INFO: `/v3/projects/0baa696bf0e14062b974d2eac6edb520` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:45.687921 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc55c9-cbbe-4f39-acf2-177e7545c1bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0baa696bf0e14062b974d2eac6edb520)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:45.690420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc55c9-cbbe-4f39-acf2-177e7545c1bf admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:45.763924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bffc55c9-cbbe-4f39-acf2-177e7545c1bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0baa696bf0e14062b974d2eac6edb520', 'request_id': 'req-bffc55c9-cbbe-4f39-acf2-177e7545c1bf'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:45.771284 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bffc55c9-cbbe-4f39-acf2-177e7545c1bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0baa696bf0e14062b974d2eac6edb520', 'request_id': 'req-bffc55c9-cbbe-4f39-acf2-177e7545c1bf'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:45.784469 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bffc55c9-cbbe-4f39-acf2-177e7545c1bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0baa696bf0e14062b974d2eac6edb520', 'request_id': 'req-bffc55c9-cbbe-4f39-acf2-177e7545c1bf'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:45.799071 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bffc55c9-cbbe-4f39-acf2-177e7545c1bf admin admin] The token cache is being invalidate because project 0baa696bf0e14062b974d2eac6edb520 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:47:45.799448 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bffc55c9-cbbe-4f39-acf2-177e7545c1bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:45.800219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bffc55c9-cbbe-4f39-acf2-177e7545c1bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:45.803824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1170/2360] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:47:45 2022] DELETE /identity/v3/projects/0baa696bf0e14062b974d2eac6edb520 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:45.808495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355ea07-cdf6-42a3-a065-da9f557330c3 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:45.809076 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355ea07-cdf6-42a3-a065-da9f557330c3 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:45.809630 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355ea07-cdf6-42a3-a065-da9f557330c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:45.837039 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-e355ea07-cdf6-42a3-a065-da9f557330c3 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:45.838455 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e355ea07-cdf6-42a3-a065-da9f557330c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:45.931046 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1191/2361] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:47:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:47:45.943836 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a5703-d00b-4dea-894e-c1bcf77fdc83 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:45.944303 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a5703-d00b-4dea-894e-c1bcf77fdc83 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:45.944756 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a5703-d00b-4dea-894e-c1bcf77fdc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:45.968706 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-874a5703-d00b-4dea-894e-c1bcf77fdc83 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:45.970015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-874a5703-d00b-4dea-894e-c1bcf77fdc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:45.978548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1171/2362] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:47:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:47:45.983746 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355ea07-cdf6-42a3-a065-da9f557330c3 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:45.994221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355ea07-cdf6-42a3-a065-da9f557330c3 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:45.999643 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51405e8-bf1b-4970-8cf9-dbba48ceabe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:46.000207 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51405e8-bf1b-4970-8cf9-dbba48ceabe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:46.000767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51405e8-bf1b-4970-8cf9-dbba48ceabe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:46.000767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51405e8-bf1b-4970-8cf9-dbba48ceabe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:46.001482 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51405e8-bf1b-4970-8cf9-dbba48ceabe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:46.001797 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51405e8-bf1b-4970-8cf9-dbba48ceabe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:46.003561 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51405e8-bf1b-4970-8cf9-dbba48ceabe7 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:46.020791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1192/2363] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:47:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:46.022082 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a5703-d00b-4dea-894e-c1bcf77fdc83 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:46.032762 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a5703-d00b-4dea-894e-c1bcf77fdc83 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:46.037805 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c810b53-18ed-4c99-a1d4-761423a7ada6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:46.038455 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c810b53-18ed-4c99-a1d4-761423a7ada6 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:46.040168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c810b53-18ed-4c99-a1d4-761423a7ada6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:46.040168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c810b53-18ed-4c99-a1d4-761423a7ada6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:46.040168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c810b53-18ed-4c99-a1d4-761423a7ada6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:46.041636 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c810b53-18ed-4c99-a1d4-761423a7ada6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:46.078863 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1172/2364] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:47:46 2022] POST /identity/v3/projects => generated 378 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:46.080148 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51405e8-bf1b-4970-8cf9-dbba48ceabe7 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:46.093150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51405e8-bf1b-4970-8cf9-dbba48ceabe7 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:46.169135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c789ddc-3fd2-414c-9fa4-f11196145196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:46.169839 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c789ddc-3fd2-414c-9fa4-f11196145196 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:46.170132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c789ddc-3fd2-414c-9fa4-f11196145196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:46.170411 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c789ddc-3fd2-414c-9fa4-f11196145196 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:46.171091 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c789ddc-3fd2-414c-9fa4-f11196145196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:46.173027 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c789ddc-3fd2-414c-9fa4-f11196145196 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:46.195595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1193/2365] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:47:46 2022] POST /identity/v3/users => generated 290 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:46.201510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c810b53-18ed-4c99-a1d4-761423a7ada6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:46.213000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c810b53-18ed-4c99-a1d4-761423a7ada6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:46.219001 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3074f1-ce3d-432f-9ebb-4b9956bc6c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:46.219666 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3074f1-ce3d-432f-9ebb-4b9956bc6c3e admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:46.219978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3074f1-ce3d-432f-9ebb-4b9956bc6c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:46.220256 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3074f1-ce3d-432f-9ebb-4b9956bc6c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:46.220965 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3074f1-ce3d-432f-9ebb-4b9956bc6c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:46.222641 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3074f1-ce3d-432f-9ebb-4b9956bc6c3e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:46.237373 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1173/2366] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:47:46 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:47:46.238740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c789ddc-3fd2-414c-9fa4-f11196145196 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:46.249361 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c789ddc-3fd2-414c-9fa4-f11196145196 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:46.254494 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc80bf6-17f8-4b9c-a631-d7e52646b42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:46.255191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc80bf6-17f8-4b9c-a631-d7e52646b42c admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:46.255473 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc80bf6-17f8-4b9c-a631-d7e52646b42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:46.256041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc80bf6-17f8-4b9c-a631-d7e52646b42c admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:46.256742 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc80bf6-17f8-4b9c-a631-d7e52646b42c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:46.258388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc80bf6-17f8-4b9c-a631-d7e52646b42c admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:46.270863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1194/2367] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:47:46 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:47:46.275101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3074f1-ce3d-432f-9ebb-4b9956bc6c3e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:46.286128 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3074f1-ce3d-432f-9ebb-4b9956bc6c3e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:46.290789 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c693a26-8004-4a37-a50f-f290611cbe23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:46.291469 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c693a26-8004-4a37-a50f-f290611cbe23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:46.291750 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c693a26-8004-4a37-a50f-f290611cbe23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:46.292028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c693a26-8004-4a37-a50f-f290611cbe23 admin admin] PATH_INFO: `/v3/projects/cb1a4988e3c442c1a89c3b5916df8855/users/9d74bb21ca2c4fabb8a875decd0f55f9/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:46.308001 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c693a26-8004-4a37-a50f-f290611cbe23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb1a4988e3c442c1a89c3b5916df8855, user_id=9d74bb21ca2c4fabb8a875decd0f55f9, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:46.310799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c693a26-8004-4a37-a50f-f290611cbe23 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:46.325571 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1174/2368] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:47:46 2022] PUT /identity/v3/projects/cb1a4988e3c442c1a89c3b5916df8855/users/9d74bb21ca2c4fabb8a875decd0f55f9/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:46.335503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ebe68-cf3c-4839-b7f7-15e573326bb2 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:46.335798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ebe68-cf3c-4839-b7f7-15e573326bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:46.336075 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ebe68-cf3c-4839-b7f7-15e573326bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:46.365024 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-de0ebe68-cf3c-4839-b7f7-15e573326bb2 None None] MFA Rules not processed for user `9d74bb21ca2c4fabb8a875decd0f55f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:46.366361 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-de0ebe68-cf3c-4839-b7f7-15e573326bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:46.393856 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:46.447201 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c693a26-8004-4a37-a50f-f290611cbe23 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:46.448202 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1195/2369] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:47:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:47:46.506877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:46.535212 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c693a26-8004-4a37-a50f-f290611cbe23 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:46.540977 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9f103-c8bf-4208-8034-68c278450172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:46.541711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9f103-c8bf-4208-8034-68c278450172 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:46.542127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9f103-c8bf-4208-8034-68c278450172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:46.542493 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9f103-c8bf-4208-8034-68c278450172 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:46.543829 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9f103-c8bf-4208-8034-68c278450172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:46.546169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9f103-c8bf-4208-8034-68c278450172 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:46.583301 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1175/2370] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:47:46 2022] POST /identity/v3/projects => generated 380 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:46.587927 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ebe68-cf3c-4839-b7f7-15e573326bb2 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:46.599860 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ebe68-cf3c-4839-b7f7-15e573326bb2 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:46.665643 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd34cb-ede2-4e58-a0ae-318956a22a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:46.666357 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd34cb-ede2-4e58-a0ae-318956a22a6a admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:46.668803 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd34cb-ede2-4e58-a0ae-318956a22a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:46.668803 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd34cb-ede2-4e58-a0ae-318956a22a6a admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:46.668803 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd34cb-ede2-4e58-a0ae-318956a22a6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:46.670615 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd34cb-ede2-4e58-a0ae-318956a22a6a admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:46.694984 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1196/2371] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:47:46 2022] POST /identity/v3/users => generated 297 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:46.699119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9f103-c8bf-4208-8034-68c278450172 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:46.710503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9f103-c8bf-4208-8034-68c278450172 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:46.716173 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bc625-9440-4e05-8173-6ae15881cfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:46.716806 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bc625-9440-4e05-8173-6ae15881cfbf admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:46.717092 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bc625-9440-4e05-8173-6ae15881cfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:46.717432 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bc625-9440-4e05-8173-6ae15881cfbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:46.718096 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167bc625-9440-4e05-8173-6ae15881cfbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:46.720132 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167bc625-9440-4e05-8173-6ae15881cfbf admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:46.732543 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1176/2372] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:47:46 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:47:46.736602 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd34cb-ede2-4e58-a0ae-318956a22a6a admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:46.747953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd34cb-ede2-4e58-a0ae-318956a22a6a admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:46.753133 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7380e94-1d50-4455-b4b4-4a25bc6b8ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:46.753935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7380e94-1d50-4455-b4b4-4a25bc6b8ff3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:46.754315 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7380e94-1d50-4455-b4b4-4a25bc6b8ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:46.754660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7380e94-1d50-4455-b4b4-4a25bc6b8ff3 admin admin] PATH_INFO: `/v3/projects/246e451758544de6b51b659b9be5aeda/users/89124b6e560d44239f24ec3eb0a7e2cb/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:46.771376 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7380e94-1d50-4455-b4b4-4a25bc6b8ff3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=246e451758544de6b51b659b9be5aeda, user_id=89124b6e560d44239f24ec3eb0a7e2cb, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:46.774483 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7380e94-1d50-4455-b4b4-4a25bc6b8ff3 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:46.790245 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1197/2373] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:47:46 2022] PUT /identity/v3/projects/246e451758544de6b51b659b9be5aeda/users/89124b6e560d44239f24ec3eb0a7e2cb/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:46.794893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bc625-9440-4e05-8173-6ae15881cfbf admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:46.804894 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bc625-9440-4e05-8173-6ae15881cfbf admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:46.871521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3e980-dddb-4cdb-97f5-a256b7125e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:46.872196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3e980-dddb-4cdb-97f5-a256b7125e24 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:46.872668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3e980-dddb-4cdb-97f5-a256b7125e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:46.872816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3e980-dddb-4cdb-97f5-a256b7125e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:46.873487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd3e980-dddb-4cdb-97f5-a256b7125e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:46.875387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd3e980-dddb-4cdb-97f5-a256b7125e24 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:46.887960 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1177/2374] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:47:46 2022] GET /identity/v3/roles => generated 1427 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:47:46.892336 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7380e94-1d50-4455-b4b4-4a25bc6b8ff3 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:46.905422 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7380e94-1d50-4455-b4b4-4a25bc6b8ff3 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:46.911679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed1db2-e5fa-482c-bcd2-f751e54cfa27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:46.912365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed1db2-e5fa-482c-bcd2-f751e54cfa27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:46.912756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed1db2-e5fa-482c-bcd2-f751e54cfa27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:46.913082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed1db2-e5fa-482c-bcd2-f751e54cfa27 admin admin] PATH_INFO: `/v3/projects/246e451758544de6b51b659b9be5aeda/users/89124b6e560d44239f24ec3eb0a7e2cb/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:46.918668 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed1db2-e5fa-482c-bcd2-f751e54cfa27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=246e451758544de6b51b659b9be5aeda, user_id=89124b6e560d44239f24ec3eb0a7e2cb, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:46.918668 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed1db2-e5fa-482c-bcd2-f751e54cfa27 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:46.930148 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1198/2375] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:47:46 2022] PUT /identity/v3/projects/246e451758544de6b51b659b9be5aeda/users/89124b6e560d44239f24ec3eb0a7e2cb/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:47:46.943027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58587a-949f-46ae-b001-e878c49de2ef None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:46.946329 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58587a-949f-46ae-b001-e878c49de2ef None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:46.949967 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58587a-949f-46ae-b001-e878c49de2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:47.004168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6f58587a-949f-46ae-b001-e878c49de2ef None None] MFA Rules not processed for user `89124b6e560d44239f24ec3eb0a7e2cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:47.006365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6f58587a-949f-46ae-b001-e878c49de2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:47.119427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1178/2376] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:47:46 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:47:47.130335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed1db2-e5fa-482c-bcd2-f751e54cfa27 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:47.132508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-99ed1db2-e5fa-482c-bcd2-f751e54cfa27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:47.214222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed1db2-e5fa-482c-bcd2-f751e54cfa27 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:47.219080 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191d644-f978-4ef7-bf95-165415bb21bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:47.219837 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191d644-f978-4ef7-bf95-165415bb21bd service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:47.220514 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191d644-f978-4ef7-bf95-165415bb21bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:47.221001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191d644-f978-4ef7-bf95-165415bb21bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:47.232965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191d644-f978-4ef7-bf95-165415bb21bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:47.234839 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191d644-f978-4ef7-bf95-165415bb21bd service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:47.249214 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1199/2377] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:47:47 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:47:47.338285 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58587a-949f-46ae-b001-e878c49de2ef None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:47.350492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58587a-949f-46ae-b001-e878c49de2ef None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:47.361302 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b4d17-69ef-4ede-a400-b6b84d96a828 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:47.362177 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b4d17-69ef-4ede-a400-b6b84d96a828 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:47.362897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b4d17-69ef-4ede-a400-b6b84d96a828 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:47.363820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b4d17-69ef-4ede-a400-b6b84d96a828 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:47.416375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:47.452744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b4d17-69ef-4ede-a400-b6b84d96a828 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:47.454411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b4d17-69ef-4ede-a400-b6b84d96a828 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:47.465856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1179/2378] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:47:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:47:47.609660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191d644-f978-4ef7-bf95-165415bb21bd service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:47.611783 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c191d644-f978-4ef7-bf95-165415bb21bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:47.655514 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:47.700629 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191d644-f978-4ef7-bf95-165415bb21bd service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:47.705283 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d411e5-1571-493c-8e1a-fd6218af0e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:47.705905 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d411e5-1571-493c-8e1a-fd6218af0e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:47.706185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d411e5-1571-493c-8e1a-fd6218af0e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:47.706488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d411e5-1571-493c-8e1a-fd6218af0e1f admin admin] PATH_INFO: `/v3/users/0e93d159e8644203808918c5ae86fc74` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:47.707775 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d411e5-1571-493c-8e1a-fd6218af0e1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e93d159e8644203808918c5ae86fc74)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:47.709800 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d411e5-1571-493c-8e1a-fd6218af0e1f admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:47.760411 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-10d411e5-1571-493c-8e1a-fd6218af0e1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e93d159e8644203808918c5ae86fc74'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:47.761069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-10d411e5-1571-493c-8e1a-fd6218af0e1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e93d159e8644203808918c5ae86fc74'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:47.769178 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-10d411e5-1571-493c-8e1a-fd6218af0e1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e93d159e8644203808918c5ae86fc74'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:47.787900 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-10d411e5-1571-493c-8e1a-fd6218af0e1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e93d159e8644203808918c5ae86fc74'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:47.795606 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-10d411e5-1571-493c-8e1a-fd6218af0e1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e93d159e8644203808918c5ae86fc74'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:47.801831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1200/2379] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:47:47 2022] DELETE /identity/v3/users/0e93d159e8644203808918c5ae86fc74 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:47.977267 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b4d17-69ef-4ede-a400-b6b84d96a828 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:47.988955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b4d17-69ef-4ede-a400-b6b84d96a828 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:48.049502 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390b1ab-3c36-46de-99a3-6850442257ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:48.050074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390b1ab-3c36-46de-99a3-6850442257ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:48.050360 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390b1ab-3c36-46de-99a3-6850442257ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:48.050642 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390b1ab-3c36-46de-99a3-6850442257ae admin admin] PATH_INFO: `/v3/projects/ada46409d8ae4abfaf19056675f7fbc6` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:48.051798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390b1ab-3c36-46de-99a3-6850442257ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=ada46409d8ae4abfaf19056675f7fbc6)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:48.053751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390b1ab-3c36-46de-99a3-6850442257ae admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:48.102213 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b390b1ab-3c36-46de-99a3-6850442257ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ada46409d8ae4abfaf19056675f7fbc6', 'request_id': 'req-b390b1ab-3c36-46de-99a3-6850442257ae'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:48.109133 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b390b1ab-3c36-46de-99a3-6850442257ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ada46409d8ae4abfaf19056675f7fbc6', 'request_id': 'req-b390b1ab-3c36-46de-99a3-6850442257ae'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:48.116913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b390b1ab-3c36-46de-99a3-6850442257ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ada46409d8ae4abfaf19056675f7fbc6', 'request_id': 'req-b390b1ab-3c36-46de-99a3-6850442257ae'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:48.127468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b390b1ab-3c36-46de-99a3-6850442257ae admin admin] The token cache is being invalidate because project ada46409d8ae4abfaf19056675f7fbc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:47:48.128338 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b390b1ab-3c36-46de-99a3-6850442257ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:48.128995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b390b1ab-3c36-46de-99a3-6850442257ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:48.130530 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1180/2380] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:47:47 2022] DELETE /identity/v3/projects/ada46409d8ae4abfaf19056675f7fbc6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:48.135901 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79658478-2791-441f-b0ec-b9d5a7eee045 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:48.136402 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79658478-2791-441f-b0ec-b9d5a7eee045 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:48.136512 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79658478-2791-441f-b0ec-b9d5a7eee045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:48.156952 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-79658478-2791-441f-b0ec-b9d5a7eee045 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:48.158370 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-79658478-2791-441f-b0ec-b9d5a7eee045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:48.222805 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1201/2381] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:47:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:47:48.232080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05458698-9a24-4c6e-986d-2779b5a6d1c6 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:48.232389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05458698-9a24-4c6e-986d-2779b5a6d1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:48.232669 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05458698-9a24-4c6e-986d-2779b5a6d1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:48.252718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-05458698-9a24-4c6e-986d-2779b5a6d1c6 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:48.253964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-05458698-9a24-4c6e-986d-2779b5a6d1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:48.262495 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1181/2382] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:47:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:47:48.267163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79658478-2791-441f-b0ec-b9d5a7eee045 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:48.276839 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79658478-2791-441f-b0ec-b9d5a7eee045 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:48.281854 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696232a1-96d4-4fa5-9a30-ee19c393f1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:48.282455 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696232a1-96d4-4fa5-9a30-ee19c393f1e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:48.282732 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696232a1-96d4-4fa5-9a30-ee19c393f1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:48.283000 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696232a1-96d4-4fa5-9a30-ee19c393f1e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:48.283681 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696232a1-96d4-4fa5-9a30-ee19c393f1e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:48.284201 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696232a1-96d4-4fa5-9a30-ee19c393f1e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:48.285744 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696232a1-96d4-4fa5-9a30-ee19c393f1e6 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:48.298624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1202/2383] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:47:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:48.303257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05458698-9a24-4c6e-986d-2779b5a6d1c6 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:48.313015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05458698-9a24-4c6e-986d-2779b5a6d1c6 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:48.317639 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca2046-0b1f-4112-a520-10cbd7e4f507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:48.318247 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca2046-0b1f-4112-a520-10cbd7e4f507 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:48.318528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca2046-0b1f-4112-a520-10cbd7e4f507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:48.318806 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca2046-0b1f-4112-a520-10cbd7e4f507 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:48.319468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca2046-0b1f-4112-a520-10cbd7e4f507 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:48.321400 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca2046-0b1f-4112-a520-10cbd7e4f507 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:48.349645 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1182/2384] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:47:48 2022] POST /identity/v3/projects => generated 356 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:48.353975 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696232a1-96d4-4fa5-9a30-ee19c393f1e6 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:48.364558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696232a1-96d4-4fa5-9a30-ee19c393f1e6 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:48.390748 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:48.423329 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1577f-8e23-4a22-a73b-f468d369bc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:48.424011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1577f-8e23-4a22-a73b-f468d369bc91 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:48.424360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1577f-8e23-4a22-a73b-f468d369bc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:48.424664 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1577f-8e23-4a22-a73b-f468d369bc91 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:48.425435 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1577f-8e23-4a22-a73b-f468d369bc91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:48.427346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1577f-8e23-4a22-a73b-f468d369bc91 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:48.448585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1203/2385] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:47:48 2022] POST /identity/v3/users => generated 279 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:48.453879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca2046-0b1f-4112-a520-10cbd7e4f507 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:48.465042 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca2046-0b1f-4112-a520-10cbd7e4f507 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:48.469970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4f284-6e85-47cd-a57f-851bc4ee60ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:48.470587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4f284-6e85-47cd-a57f-851bc4ee60ee admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:48.470894 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4f284-6e85-47cd-a57f-851bc4ee60ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:48.471175 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4f284-6e85-47cd-a57f-851bc4ee60ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:48.471870 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4f284-6e85-47cd-a57f-851bc4ee60ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:48.473521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4f284-6e85-47cd-a57f-851bc4ee60ee admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:48.483622 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1183/2386] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:47:48 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:47:48.487898 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1577f-8e23-4a22-a73b-f468d369bc91 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:48.498075 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1577f-8e23-4a22-a73b-f468d369bc91 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:48.502737 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b1af1-8805-49e0-b1a8-6b2745fec0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:48.503349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b1af1-8805-49e0-b1a8-6b2745fec0ed admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:48.503663 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b1af1-8805-49e0-b1a8-6b2745fec0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:48.503944 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b1af1-8805-49e0-b1a8-6b2745fec0ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:48.504635 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b1af1-8805-49e0-b1a8-6b2745fec0ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:48.506552 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b1af1-8805-49e0-b1a8-6b2745fec0ed admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:48.517218 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1204/2387] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:47:48 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:47:48.521327 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4f284-6e85-47cd-a57f-851bc4ee60ee admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:48.532295 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4f284-6e85-47cd-a57f-851bc4ee60ee admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:48.536985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991b844-5946-4b51-9ccb-a7457220bd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:48.537702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991b844-5946-4b51-9ccb-a7457220bd3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:48.538016 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991b844-5946-4b51-9ccb-a7457220bd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:48.538297 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991b844-5946-4b51-9ccb-a7457220bd3a admin admin] PATH_INFO: `/v3/projects/fec27884e3dc476bb8d0c46d0b33c1d9/users/fa6a3343bb7b4e9f9327bbcbebea5ebe/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:48.553233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6991b844-5946-4b51-9ccb-a7457220bd3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fec27884e3dc476bb8d0c46d0b33c1d9, user_id=fa6a3343bb7b4e9f9327bbcbebea5ebe, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:48.555934 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6991b844-5946-4b51-9ccb-a7457220bd3a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:48.565091 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:48.570334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1184/2388] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:47:48 2022] PUT /identity/v3/projects/fec27884e3dc476bb8d0c46d0b33c1d9/users/fa6a3343bb7b4e9f9327bbcbebea5ebe/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:48.583191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b6434-8d5b-4093-9934-12a0810a56cf None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:48.583480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b6434-8d5b-4093-9934-12a0810a56cf None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:48.583757 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b6434-8d5b-4093-9934-12a0810a56cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:48.611829 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-621b6434-8d5b-4093-9934-12a0810a56cf None None] MFA Rules not processed for user `fa6a3343bb7b4e9f9327bbcbebea5ebe`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:48.613064 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-621b6434-8d5b-4093-9934-12a0810a56cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:48.687814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1205/2389] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:47:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:47:48.692669 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991b844-5946-4b51-9ccb-a7457220bd3a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:48.767098 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991b844-5946-4b51-9ccb-a7457220bd3a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:48.771838 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1767fc-5947-4999-b908-bd5945f57180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:48.772460 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1767fc-5947-4999-b908-bd5945f57180 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:48.772748 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1767fc-5947-4999-b908-bd5945f57180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:48.773035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1767fc-5947-4999-b908-bd5945f57180 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:48.773856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1767fc-5947-4999-b908-bd5945f57180 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:48.775696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1767fc-5947-4999-b908-bd5945f57180 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:48.805702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1185/2390] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:47:48 2022] POST /identity/v3/projects => generated 358 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:48.810972 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b6434-8d5b-4093-9934-12a0810a56cf None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:48.823427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b6434-8d5b-4093-9934-12a0810a56cf None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:48.920103 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc16af3-626d-477c-926e-686c446c2062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:48.920103 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc16af3-626d-477c-926e-686c446c2062 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:48.920103 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc16af3-626d-477c-926e-686c446c2062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:48.920103 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc16af3-626d-477c-926e-686c446c2062 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:48.920652 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc16af3-626d-477c-926e-686c446c2062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:48.920652 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc16af3-626d-477c-926e-686c446c2062 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:48.948413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1206/2391] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:47:48 2022] POST /identity/v3/users => generated 286 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:48.953135 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1767fc-5947-4999-b908-bd5945f57180 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:48.977309 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1767fc-5947-4999-b908-bd5945f57180 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:48.993496 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7657a-29ad-4e5f-b5f2-e311486e6735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:48.994401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7657a-29ad-4e5f-b5f2-e311486e6735 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:48.994888 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7657a-29ad-4e5f-b5f2-e311486e6735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:48.995368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7657a-29ad-4e5f-b5f2-e311486e6735 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:48.996787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f7657a-29ad-4e5f-b5f2-e311486e6735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:48.998787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f7657a-29ad-4e5f-b5f2-e311486e6735 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:49.026722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1186/2392] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:47:48 2022] GET /identity/v3/roles => generated 1427 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:47:49.031120 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc16af3-626d-477c-926e-686c446c2062 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:49.042115 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc16af3-626d-477c-926e-686c446c2062 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:49.048558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aad403-f6c8-4e10-9934-1b88c116b2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:49.049288 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aad403-f6c8-4e10-9934-1b88c116b2da admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:49.049718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aad403-f6c8-4e10-9934-1b88c116b2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:49.050033 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aad403-f6c8-4e10-9934-1b88c116b2da admin admin] PATH_INFO: `/v3/projects/5d49a05313be4b38b53d09efe9f7683a/users/88b21ff05f2d467d84415c0bad4a0ae9/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:49.068223 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aad403-f6c8-4e10-9934-1b88c116b2da admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d49a05313be4b38b53d09efe9f7683a, user_id=88b21ff05f2d467d84415c0bad4a0ae9, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:49.072846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aad403-f6c8-4e10-9934-1b88c116b2da admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:49.088661 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1207/2393] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:47:49 2022] PUT /identity/v3/projects/5d49a05313be4b38b53d09efe9f7683a/users/88b21ff05f2d467d84415c0bad4a0ae9/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:49.093309 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7657a-29ad-4e5f-b5f2-e311486e6735 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:49.107192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7657a-29ad-4e5f-b5f2-e311486e6735 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:49.165594 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6041fa-6eb2-4f43-9a83-2b39790c48a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:49.166221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6041fa-6eb2-4f43-9a83-2b39790c48a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:49.166533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6041fa-6eb2-4f43-9a83-2b39790c48a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:49.166809 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6041fa-6eb2-4f43-9a83-2b39790c48a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:49.168101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6041fa-6eb2-4f43-9a83-2b39790c48a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:49.169849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6041fa-6eb2-4f43-9a83-2b39790c48a3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:49.179802 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1187/2394] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:47:49 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:47:49.185312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aad403-f6c8-4e10-9934-1b88c116b2da admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:49.191689 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:49.197993 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aad403-f6c8-4e10-9934-1b88c116b2da admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:49.203010 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b2443-1e08-46a2-aac6-1b554aefeab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:49.203674 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b2443-1e08-46a2-aac6-1b554aefeab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:49.203960 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b2443-1e08-46a2-aac6-1b554aefeab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:49.204241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b2443-1e08-46a2-aac6-1b554aefeab2 admin admin] PATH_INFO: `/v3/projects/5d49a05313be4b38b53d09efe9f7683a/users/88b21ff05f2d467d84415c0bad4a0ae9/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:49.206213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b2443-1e08-46a2-aac6-1b554aefeab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d49a05313be4b38b53d09efe9f7683a, user_id=88b21ff05f2d467d84415c0bad4a0ae9, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:49.208859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b2443-1e08-46a2-aac6-1b554aefeab2 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:49.220577 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1208/2395] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:47:49 2022] PUT /identity/v3/projects/5d49a05313be4b38b53d09efe9f7683a/users/88b21ff05f2d467d84415c0bad4a0ae9/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:49.287531 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12f13c-2fb8-44a0-a840-b4088f49ce76 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:49.287826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12f13c-2fb8-44a0-a840-b4088f49ce76 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:49.288136 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12f13c-2fb8-44a0-a840-b4088f49ce76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:49.319922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-7e12f13c-2fb8-44a0-a840-b4088f49ce76 None None] MFA Rules not processed for user `88b21ff05f2d467d84415c0bad4a0ae9`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:49.321081 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7e12f13c-2fb8-44a0-a840-b4088f49ce76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:49.385501 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:49.398105 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1188/2396] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:47:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:47:49.410205 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b2443-1e08-46a2-aac6-1b554aefeab2 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:49.411927 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-703b2443-1e08-46a2-aac6-1b554aefeab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:49.476913 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b2443-1e08-46a2-aac6-1b554aefeab2 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:49.481804 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fae6c8-e9a6-4e3f-905a-a6714a0b82a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:49.482430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fae6c8-e9a6-4e3f-905a-a6714a0b82a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:49.482713 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fae6c8-e9a6-4e3f-905a-a6714a0b82a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:49.482987 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fae6c8-e9a6-4e3f-905a-a6714a0b82a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:49.492900 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fae6c8-e9a6-4e3f-905a-a6714a0b82a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:49.494417 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fae6c8-e9a6-4e3f-905a-a6714a0b82a5 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:49.504450 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1209/2397] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:47:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:47:49.592151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12f13c-2fb8-44a0-a840-b4088f49ce76 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:49.603842 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12f13c-2fb8-44a0-a840-b4088f49ce76 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:49.609079 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36759a9e-96cb-411b-b88c-e909e033498a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:49.609735 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36759a9e-96cb-411b-b88c-e909e033498a service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:49.610021 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36759a9e-96cb-411b-b88c-e909e033498a service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:49.610299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36759a9e-96cb-411b-b88c-e909e033498a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:49.673545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36759a9e-96cb-411b-b88c-e909e033498a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:49.675008 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36759a9e-96cb-411b-b88c-e909e033498a service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:49.684390 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1189/2398] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:47:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:47:56.545630 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fae6c8-e9a6-4e3f-905a-a6714a0b82a5 service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:56.547166 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-74fae6c8-e9a6-4e3f-905a-a6714a0b82a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:56.613314 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fae6c8-e9a6-4e3f-905a-a6714a0b82a5 service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:56.617776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fd1d7-9c5e-42a7-ba67-601dab2b90f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:56.618412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fd1d7-9c5e-42a7-ba67-601dab2b90f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:56.618691 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fd1d7-9c5e-42a7-ba67-601dab2b90f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:56.618980 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fd1d7-9c5e-42a7-ba67-601dab2b90f4 admin admin] PATH_INFO: `/v3/users/9d74bb21ca2c4fabb8a875decd0f55f9` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:56.620105 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363fd1d7-9c5e-42a7-ba67-601dab2b90f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d74bb21ca2c4fabb8a875decd0f55f9)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:56.622352 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363fd1d7-9c5e-42a7-ba67-601dab2b90f4 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:56.672804 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-363fd1d7-9c5e-42a7-ba67-601dab2b90f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d74bb21ca2c4fabb8a875decd0f55f9'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:56.673640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-363fd1d7-9c5e-42a7-ba67-601dab2b90f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d74bb21ca2c4fabb8a875decd0f55f9'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:56.681071 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-363fd1d7-9c5e-42a7-ba67-601dab2b90f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d74bb21ca2c4fabb8a875decd0f55f9'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:56.693266 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-363fd1d7-9c5e-42a7-ba67-601dab2b90f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d74bb21ca2c4fabb8a875decd0f55f9'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:56.701172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-363fd1d7-9c5e-42a7-ba67-601dab2b90f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d74bb21ca2c4fabb8a875decd0f55f9'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:56.707598 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1210/2399] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:47:56 2022] DELETE /identity/v3/users/9d74bb21ca2c4fabb8a875decd0f55f9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:56.712234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069ce55-92fa-4800-a4c4-9ac1a3caf1c0 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:56.712732 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069ce55-92fa-4800-a4c4-9ac1a3caf1c0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:56.712856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069ce55-92fa-4800-a4c4-9ac1a3caf1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:56.734432 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e069ce55-92fa-4800-a4c4-9ac1a3caf1c0 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:56.735662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e069ce55-92fa-4800-a4c4-9ac1a3caf1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:56.798156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1190/2400] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:47:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:47:56.808447 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fd1d7-9c5e-42a7-ba67-601dab2b90f4 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:56.809968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-363fd1d7-9c5e-42a7-ba67-601dab2b90f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:56.824429 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:56.873836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fd1d7-9c5e-42a7-ba67-601dab2b90f4 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:56.877537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f0f63-dc94-401b-8623-961a73bbd055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:56.878160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f0f63-dc94-401b-8623-961a73bbd055 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:56.878447 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f0f63-dc94-401b-8623-961a73bbd055 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:56.878727 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f0f63-dc94-401b-8623-961a73bbd055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:56.888791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f0f63-dc94-401b-8623-961a73bbd055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:56.890270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f0f63-dc94-401b-8623-961a73bbd055 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:56.899921 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1211/2401] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:47:56 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:47:57.188090 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e069ce55-92fa-4800-a4c4-9ac1a3caf1c0 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:57.189781 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e069ce55-92fa-4800-a4c4-9ac1a3caf1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:57.204947 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e069ce55-92fa-4800-a4c4-9ac1a3caf1c0 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:57.220032 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946ce2f-55bd-452a-baff-c0d04adfacc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:57.220928 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946ce2f-55bd-452a-baff-c0d04adfacc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:57.221412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946ce2f-55bd-452a-baff-c0d04adfacc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:57.221792 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946ce2f-55bd-452a-baff-c0d04adfacc4 admin admin] PATH_INFO: `/v3/projects/cb1a4988e3c442c1a89c3b5916df8855` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:57.223258 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a946ce2f-55bd-452a-baff-c0d04adfacc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb1a4988e3c442c1a89c3b5916df8855)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:57.225493 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a946ce2f-55bd-452a-baff-c0d04adfacc4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:57.282857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a946ce2f-55bd-452a-baff-c0d04adfacc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb1a4988e3c442c1a89c3b5916df8855', 'request_id': 'req-a946ce2f-55bd-452a-baff-c0d04adfacc4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:57.289973 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a946ce2f-55bd-452a-baff-c0d04adfacc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb1a4988e3c442c1a89c3b5916df8855', 'request_id': 'req-a946ce2f-55bd-452a-baff-c0d04adfacc4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:57.304738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a946ce2f-55bd-452a-baff-c0d04adfacc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb1a4988e3c442c1a89c3b5916df8855', 'request_id': 'req-a946ce2f-55bd-452a-baff-c0d04adfacc4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:57.317968 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a946ce2f-55bd-452a-baff-c0d04adfacc4 admin admin] The token cache is being invalidate because project cb1a4988e3c442c1a89c3b5916df8855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:47:57.318342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a946ce2f-55bd-452a-baff-c0d04adfacc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:57.319061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a946ce2f-55bd-452a-baff-c0d04adfacc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:57.320678 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1191/2402] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:47:57 2022] DELETE /identity/v3/projects/cb1a4988e3c442c1a89c3b5916df8855 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:57.325543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f0f63-dc94-401b-8623-961a73bbd055 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:57.336078 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f0f63-dc94-401b-8623-961a73bbd055 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:57.408233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3e77b-c83b-45cc-abaa-a91e30c6d0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:57.408882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3e77b-c83b-45cc-abaa-a91e30c6d0ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:57.409251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3e77b-c83b-45cc-abaa-a91e30c6d0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:57.409530 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3e77b-c83b-45cc-abaa-a91e30c6d0ea admin admin] PATH_INFO: `/v3/users/89124b6e560d44239f24ec3eb0a7e2cb` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:57.410679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3e77b-c83b-45cc-abaa-a91e30c6d0ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=89124b6e560d44239f24ec3eb0a7e2cb)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:57.412718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3e77b-c83b-45cc-abaa-a91e30c6d0ea admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:57.463309 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-05f3e77b-c83b-45cc-abaa-a91e30c6d0ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89124b6e560d44239f24ec3eb0a7e2cb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:57.463993 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-05f3e77b-c83b-45cc-abaa-a91e30c6d0ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89124b6e560d44239f24ec3eb0a7e2cb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:57.472511 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-05f3e77b-c83b-45cc-abaa-a91e30c6d0ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89124b6e560d44239f24ec3eb0a7e2cb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:57.488182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-05f3e77b-c83b-45cc-abaa-a91e30c6d0ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89124b6e560d44239f24ec3eb0a7e2cb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:57.495258 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-05f3e77b-c83b-45cc-abaa-a91e30c6d0ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89124b6e560d44239f24ec3eb0a7e2cb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:57.502979 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1212/2403] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:47:57 2022] DELETE /identity/v3/users/89124b6e560d44239f24ec3eb0a7e2cb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:57.820557 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946ce2f-55bd-452a-baff-c0d04adfacc4 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:57.822031 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a946ce2f-55bd-452a-baff-c0d04adfacc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:57.836096 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946ce2f-55bd-452a-baff-c0d04adfacc4 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:57.851069 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:57.898072 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25113ae5-5e40-494f-b0e4-c00519c83963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:57.898712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25113ae5-5e40-494f-b0e4-c00519c83963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:57.899004 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25113ae5-5e40-494f-b0e4-c00519c83963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:57.899284 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25113ae5-5e40-494f-b0e4-c00519c83963 admin admin] PATH_INFO: `/v3/projects/246e451758544de6b51b659b9be5aeda` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:57.900439 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25113ae5-5e40-494f-b0e4-c00519c83963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=246e451758544de6b51b659b9be5aeda)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:57.902912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25113ae5-5e40-494f-b0e4-c00519c83963 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:57.950172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-25113ae5-5e40-494f-b0e4-c00519c83963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '246e451758544de6b51b659b9be5aeda', 'request_id': 'req-25113ae5-5e40-494f-b0e4-c00519c83963'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:57.957031 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-25113ae5-5e40-494f-b0e4-c00519c83963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '246e451758544de6b51b659b9be5aeda', 'request_id': 'req-25113ae5-5e40-494f-b0e4-c00519c83963'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:57.966382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-25113ae5-5e40-494f-b0e4-c00519c83963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '246e451758544de6b51b659b9be5aeda', 'request_id': 'req-25113ae5-5e40-494f-b0e4-c00519c83963'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:57.979530 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-25113ae5-5e40-494f-b0e4-c00519c83963 admin admin] The token cache is being invalidate because project 246e451758544de6b51b659b9be5aeda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:47:57.980571 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-25113ae5-5e40-494f-b0e4-c00519c83963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:57.981408 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-25113ae5-5e40-494f-b0e4-c00519c83963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:57.982952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1192/2404] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:47:57 2022] DELETE /identity/v3/projects/246e451758544de6b51b659b9be5aeda => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:57.987819 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c2c4f-26b5-41f4-933b-89aaedd150df None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:57.988460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c2c4f-26b5-41f4-933b-89aaedd150df None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:57.988460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c2c4f-26b5-41f4-933b-89aaedd150df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.009425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-2d2c2c4f-26b5-41f4-933b-89aaedd150df None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:58.010258 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-2d2c2c4f-26b5-41f4-933b-89aaedd150df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:58.041826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:58.074197 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1213/2405] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:47:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:47:58.081900 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdaad1b-dbe8-436f-b50a-b59e95f2bbf7 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.082194 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdaad1b-dbe8-436f-b50a-b59e95f2bbf7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.082476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdaad1b-dbe8-436f-b50a-b59e95f2bbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.102732 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-7bdaad1b-dbe8-436f-b50a-b59e95f2bbf7 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:58.103942 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7bdaad1b-dbe8-436f-b50a-b59e95f2bbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:58.112663 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1193/2406] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:47:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:47:58.116925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c2c4f-26b5-41f4-933b-89aaedd150df None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:58.126714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c2c4f-26b5-41f4-933b-89aaedd150df None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:58.131558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574a58f-d168-4f98-bdb2-48e2dbe32744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:58.132161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574a58f-d168-4f98-bdb2-48e2dbe32744 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.132446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574a58f-d168-4f98-bdb2-48e2dbe32744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.132723 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574a58f-d168-4f98-bdb2-48e2dbe32744 admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.133450 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574a58f-d168-4f98-bdb2-48e2dbe32744 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:58.133760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574a58f-d168-4f98-bdb2-48e2dbe32744 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:58.135361 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574a58f-d168-4f98-bdb2-48e2dbe32744 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:58.147925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1214/2407] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:47:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:58.152816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdaad1b-dbe8-436f-b50a-b59e95f2bbf7 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:58.162114 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdaad1b-dbe8-436f-b50a-b59e95f2bbf7 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:58.167024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ce850-6f0b-4167-b669-fccf3910b85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:58.167799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ce850-6f0b-4167-b669-fccf3910b85a admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.168083 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ce850-6f0b-4167-b669-fccf3910b85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.168363 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ce850-6f0b-4167-b669-fccf3910b85a admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.169030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ce850-6f0b-4167-b669-fccf3910b85a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:58.170966 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ce850-6f0b-4167-b669-fccf3910b85a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:58.198203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1194/2408] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:47:58 2022] POST /identity/v3/projects => generated 366 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:58.203011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574a58f-d168-4f98-bdb2-48e2dbe32744 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:58.212893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574a58f-d168-4f98-bdb2-48e2dbe32744 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:58.270448 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969cc4f-ec34-4fc3-97ca-51e66b3298e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:58.271070 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969cc4f-ec34-4fc3-97ca-51e66b3298e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.271352 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969cc4f-ec34-4fc3-97ca-51e66b3298e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.271631 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969cc4f-ec34-4fc3-97ca-51e66b3298e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.272294 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969cc4f-ec34-4fc3-97ca-51e66b3298e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:58.274306 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969cc4f-ec34-4fc3-97ca-51e66b3298e5 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:58.297494 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1215/2409] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:47:58 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:58.302251 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ce850-6f0b-4167-b669-fccf3910b85a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:58.312699 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ce850-6f0b-4167-b669-fccf3910b85a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:58.317451 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e081c32-803e-4b66-94b3-bf9dcd91dbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:58.318068 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e081c32-803e-4b66-94b3-bf9dcd91dbc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.318347 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e081c32-803e-4b66-94b3-bf9dcd91dbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.318619 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e081c32-803e-4b66-94b3-bf9dcd91dbc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.319307 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e081c32-803e-4b66-94b3-bf9dcd91dbc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:58.320869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e081c32-803e-4b66-94b3-bf9dcd91dbc7 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:58.331476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1195/2410] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:47:58 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:47:58.335519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969cc4f-ec34-4fc3-97ca-51e66b3298e5 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:58.346393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969cc4f-ec34-4fc3-97ca-51e66b3298e5 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:58.351397 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bfdad6-af74-444b-aebb-691d85cadef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:58.352009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfdad6-af74-444b-aebb-691d85cadef0 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.352297 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfdad6-af74-444b-aebb-691d85cadef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.352571 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfdad6-af74-444b-aebb-691d85cadef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.353292 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bfdad6-af74-444b-aebb-691d85cadef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:58.355800 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bfdad6-af74-444b-aebb-691d85cadef0 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:58.369307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1216/2411] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:47:58 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:47:58.373606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e081c32-803e-4b66-94b3-bf9dcd91dbc7 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:58.384039 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e081c32-803e-4b66-94b3-bf9dcd91dbc7 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:58.389656 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118bf26-9e29-4e2a-8ea2-08a6e3c88e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:58.390319 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118bf26-9e29-4e2a-8ea2-08a6e3c88e3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.390605 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118bf26-9e29-4e2a-8ea2-08a6e3c88e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.390887 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118bf26-9e29-4e2a-8ea2-08a6e3c88e3f admin admin] PATH_INFO: `/v3/projects/bd83723a36894a23b1f6ed622adae469/users/96e2fc0a08a742fd8b82339a9c62f7dc/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.406534 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118bf26-9e29-4e2a-8ea2-08a6e3c88e3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd83723a36894a23b1f6ed622adae469, user_id=96e2fc0a08a742fd8b82339a9c62f7dc, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:58.409216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118bf26-9e29-4e2a-8ea2-08a6e3c88e3f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:58.422274 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1196/2412] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:47:58 2022] PUT /identity/v3/projects/bd83723a36894a23b1f6ed622adae469/users/96e2fc0a08a742fd8b82339a9c62f7dc/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:58.432542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb29d7f-ae2f-454b-b01c-b81d5d265043 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.433005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb29d7f-ae2f-454b-b01c-b81d5d265043 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.433459 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb29d7f-ae2f-454b-b01c-b81d5d265043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.461551 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-2cb29d7f-ae2f-454b-b01c-b81d5d265043 None None] MFA Rules not processed for user `96e2fc0a08a742fd8b82339a9c62f7dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:58.462759 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-2cb29d7f-ae2f-454b-b01c-b81d5d265043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:58.528022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1217/2413] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:47:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:47:58.532592 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118bf26-9e29-4e2a-8ea2-08a6e3c88e3f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:58.578977 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:58.596361 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118bf26-9e29-4e2a-8ea2-08a6e3c88e3f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:58.601216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e458b-6e76-479e-a718-605b51536f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:58.601838 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e458b-6e76-479e-a718-605b51536f28 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.602122 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e458b-6e76-479e-a718-605b51536f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.602400 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e458b-6e76-479e-a718-605b51536f28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.603073 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e458b-6e76-479e-a718-605b51536f28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:58.605030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e458b-6e76-479e-a718-605b51536f28 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:58.631621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1197/2414] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:47:58 2022] POST /identity/v3/projects => generated 364 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:47:58.636638 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb29d7f-ae2f-454b-b01c-b81d5d265043 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:58.647482 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb29d7f-ae2f-454b-b01c-b81d5d265043 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:58.672890 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:58.705230 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9f14b-cb44-4616-a721-4c93afe24552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:58.705852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9f14b-cb44-4616-a721-4c93afe24552 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.706130 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9f14b-cb44-4616-a721-4c93afe24552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.706397 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9f14b-cb44-4616-a721-4c93afe24552 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.707062 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9f14b-cb44-4616-a721-4c93afe24552 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:58.708969 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9f14b-cb44-4616-a721-4c93afe24552 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:58.733684 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1218/2415] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:47:58 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:58.735147 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e458b-6e76-479e-a718-605b51536f28 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:58.750679 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e458b-6e76-479e-a718-605b51536f28 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:58.756004 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0a6e9-154d-40ad-a3bb-38bc87e2f921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:58.756634 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0a6e9-154d-40ad-a3bb-38bc87e2f921 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.756923 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0a6e9-154d-40ad-a3bb-38bc87e2f921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.757243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0a6e9-154d-40ad-a3bb-38bc87e2f921 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.757946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0a6e9-154d-40ad-a3bb-38bc87e2f921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:58.759560 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0a6e9-154d-40ad-a3bb-38bc87e2f921 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:58.771732 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1198/2416] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:47:58 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:47:58.776058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9f14b-cb44-4616-a721-4c93afe24552 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:58.786423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9f14b-cb44-4616-a721-4c93afe24552 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:58.791245 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d60be8-0f9c-49ac-9326-aaa9ee32e64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:58.791930 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d60be8-0f9c-49ac-9326-aaa9ee32e64f admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.792364 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d60be8-0f9c-49ac-9326-aaa9ee32e64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.792663 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d60be8-0f9c-49ac-9326-aaa9ee32e64f admin admin] PATH_INFO: `/v3/projects/56f01f575abc448e8b2f660add386da7/users/a8130f02a99048789abe30df533fcad1/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.808802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d60be8-0f9c-49ac-9326-aaa9ee32e64f admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f01f575abc448e8b2f660add386da7, user_id=a8130f02a99048789abe30df533fcad1, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:58.811495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d60be8-0f9c-49ac-9326-aaa9ee32e64f admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:58.827223 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1219/2417] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:47:58 2022] PUT /identity/v3/projects/56f01f575abc448e8b2f660add386da7/users/a8130f02a99048789abe30df533fcad1/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:47:58.831521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0a6e9-154d-40ad-a3bb-38bc87e2f921 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:58.841989 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0a6e9-154d-40ad-a3bb-38bc87e2f921 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:58.907742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484697d8-3b5b-4f21-924f-7b182db95c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:58.908906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484697d8-3b5b-4f21-924f-7b182db95c39 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.909284 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484697d8-3b5b-4f21-924f-7b182db95c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.909530 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484697d8-3b5b-4f21-924f-7b182db95c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.910771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484697d8-3b5b-4f21-924f-7b182db95c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:58.912411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484697d8-3b5b-4f21-924f-7b182db95c39 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:58.925405 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1199/2418] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:47:58 2022] GET /identity/v3/roles => generated 1427 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:47:58.930187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d60be8-0f9c-49ac-9326-aaa9ee32e64f admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:58.941779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d60be8-0f9c-49ac-9326-aaa9ee32e64f admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:58.946824 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d14cc4-8aeb-4a91-a8d7-4285d30f3b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:58.947629 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d14cc4-8aeb-4a91-a8d7-4285d30f3b2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.947915 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d14cc4-8aeb-4a91-a8d7-4285d30f3b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.948242 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d14cc4-8aeb-4a91-a8d7-4285d30f3b2a admin admin] PATH_INFO: `/v3/projects/56f01f575abc448e8b2f660add386da7/users/a8130f02a99048789abe30df533fcad1/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.950739 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d14cc4-8aeb-4a91-a8d7-4285d30f3b2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f01f575abc448e8b2f660add386da7, user_id=a8130f02a99048789abe30df533fcad1, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:58.953495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d14cc4-8aeb-4a91-a8d7-4285d30f3b2a admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:58.964329 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1220/2419] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:47:58 2022] PUT /identity/v3/projects/56f01f575abc448e8b2f660add386da7/users/a8130f02a99048789abe30df533fcad1/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:58.976429 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8c5a7-4e46-4e66-b4cc-ada0e3c2bb27 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.976949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8c5a7-4e46-4e66-b4cc-ada0e3c2bb27 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.977282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8c5a7-4e46-4e66-b4cc-ada0e3c2bb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:59.009358 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-63c8c5a7-4e46-4e66-b4cc-ada0e3c2bb27 None None] MFA Rules not processed for user `a8130f02a99048789abe30df533fcad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:59.010631 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-63c8c5a7-4e46-4e66-b4cc-ada0e3c2bb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:59.119005 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1200/2420] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:47:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:47:59.132318 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d14cc4-8aeb-4a91-a8d7-4285d30f3b2a admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:59.134418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-04d14cc4-8aeb-4a91-a8d7-4285d30f3b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:59.191650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d14cc4-8aeb-4a91-a8d7-4285d30f3b2a admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:59.195334 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f993206-5c1c-44ba-ac58-65e2411d3d9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:59.195949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f993206-5c1c-44ba-ac58-65e2411d3d9b service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:59.196235 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f993206-5c1c-44ba-ac58-65e2411d3d9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:59.196512 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f993206-5c1c-44ba-ac58-65e2411d3d9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:59.205724 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f993206-5c1c-44ba-ac58-65e2411d3d9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:59.207169 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f993206-5c1c-44ba-ac58-65e2411d3d9b service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:59.216623 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1221/2421] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:47:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:47:59.648396 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8c5a7-4e46-4e66-b4cc-ada0e3c2bb27 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:59.690978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:59.712484 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8c5a7-4e46-4e66-b4cc-ada0e3c2bb27 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:59.717660 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4738f3-954b-4bb0-9635-f17b4be08b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:59.718326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4738f3-954b-4bb0-9635-f17b4be08b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:59.718597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4738f3-954b-4bb0-9635-f17b4be08b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:59.718959 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4738f3-954b-4bb0-9635-f17b4be08b26 admin admin] PATH_INFO: `/v3/users/96e2fc0a08a742fd8b82339a9c62f7dc` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:59.720122 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4738f3-954b-4bb0-9635-f17b4be08b26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96e2fc0a08a742fd8b82339a9c62f7dc)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:59.722076 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4738f3-954b-4bb0-9635-f17b4be08b26 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:59.769001 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ae4738f3-954b-4bb0-9635-f17b4be08b26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96e2fc0a08a742fd8b82339a9c62f7dc'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:59.776278 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ae4738f3-954b-4bb0-9635-f17b4be08b26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96e2fc0a08a742fd8b82339a9c62f7dc'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:59.778202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ae4738f3-954b-4bb0-9635-f17b4be08b26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96e2fc0a08a742fd8b82339a9c62f7dc'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:59.785712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ae4738f3-954b-4bb0-9635-f17b4be08b26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96e2fc0a08a742fd8b82339a9c62f7dc'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:59.786331 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ae4738f3-954b-4bb0-9635-f17b4be08b26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96e2fc0a08a742fd8b82339a9c62f7dc'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:47:59.803758 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1201/2422] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:47:59 2022] DELETE /identity/v3/users/96e2fc0a08a742fd8b82339a9c62f7dc => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:59.809247 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00469e48-4d02-45e9-8e9d-56774cdab02e None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:59.809542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00469e48-4d02-45e9-8e9d-56774cdab02e None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:59.809852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00469e48-4d02-45e9-8e9d-56774cdab02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:59.831365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-00469e48-4d02-45e9-8e9d-56774cdab02e None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:59.832623 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-00469e48-4d02-45e9-8e9d-56774cdab02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:59.894170 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1222/2423] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:47:59 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:47:59.904953 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4738f3-954b-4bb0-9635-f17b4be08b26 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:59.906396 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ae4738f3-954b-4bb0-9635-f17b4be08b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:59.965594 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4738f3-954b-4bb0-9635-f17b4be08b26 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:59.969227 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b09050-6992-40b2-a86c-f277f2283d84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:59.969842 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b09050-6992-40b2-a86c-f277f2283d84 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:59.970122 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b09050-6992-40b2-a86c-f277f2283d84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:59.970398 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b09050-6992-40b2-a86c-f277f2283d84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:59.979372 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b09050-6992-40b2-a86c-f277f2283d84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:59.980821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b09050-6992-40b2-a86c-f277f2283d84 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:59.990362 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1202/2424] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:47:59 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:48:00.269135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00469e48-4d02-45e9-8e9d-56774cdab02e None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:00.270671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-00469e48-4d02-45e9-8e9d-56774cdab02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:00.283716 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00469e48-4d02-45e9-8e9d-56774cdab02e None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:00.288317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8c02b-dde7-4df4-99b8-3c8a9a7de47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:00.288975 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8c02b-dde7-4df4-99b8-3c8a9a7de47a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:00.289339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8c02b-dde7-4df4-99b8-3c8a9a7de47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:00.289583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8c02b-dde7-4df4-99b8-3c8a9a7de47a admin admin] PATH_INFO: `/v3/projects/bd83723a36894a23b1f6ed622adae469` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:00.290758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8c02b-dde7-4df4-99b8-3c8a9a7de47a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd83723a36894a23b1f6ed622adae469)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:00.292722 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8c02b-dde7-4df4-99b8-3c8a9a7de47a admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:00.318470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:00.349518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5eb8c02b-dde7-4df4-99b8-3c8a9a7de47a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd83723a36894a23b1f6ed622adae469', 'request_id': 'req-5eb8c02b-dde7-4df4-99b8-3c8a9a7de47a'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:00.356858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5eb8c02b-dde7-4df4-99b8-3c8a9a7de47a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd83723a36894a23b1f6ed622adae469', 'request_id': 'req-5eb8c02b-dde7-4df4-99b8-3c8a9a7de47a'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:00.365206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5eb8c02b-dde7-4df4-99b8-3c8a9a7de47a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd83723a36894a23b1f6ed622adae469', 'request_id': 'req-5eb8c02b-dde7-4df4-99b8-3c8a9a7de47a'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:00.380936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5eb8c02b-dde7-4df4-99b8-3c8a9a7de47a admin admin] The token cache is being invalidate because project bd83723a36894a23b1f6ed622adae469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:48:00.381361 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5eb8c02b-dde7-4df4-99b8-3c8a9a7de47a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:00.382083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5eb8c02b-dde7-4df4-99b8-3c8a9a7de47a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:00.383847 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1223/2425] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:48:00 2022] DELETE /identity/v3/projects/bd83723a36894a23b1f6ed622adae469 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:00.387759 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b09050-6992-40b2-a86c-f277f2283d84 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:00.406599 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b09050-6992-40b2-a86c-f277f2283d84 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:00.487101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39a2c2-2871-49b5-86be-a063790ac7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:00.488353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39a2c2-2871-49b5-86be-a063790ac7c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:00.488714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39a2c2-2871-49b5-86be-a063790ac7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:00.489064 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39a2c2-2871-49b5-86be-a063790ac7c5 admin admin] PATH_INFO: `/v3/users/a8130f02a99048789abe30df533fcad1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:00.490419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39a2c2-2871-49b5-86be-a063790ac7c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8130f02a99048789abe30df533fcad1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:00.492715 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39a2c2-2871-49b5-86be-a063790ac7c5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:00.496862 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:00.561349 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7b39a2c2-2871-49b5-86be-a063790ac7c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8130f02a99048789abe30df533fcad1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:00.569555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7b39a2c2-2871-49b5-86be-a063790ac7c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8130f02a99048789abe30df533fcad1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:00.570272 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7b39a2c2-2871-49b5-86be-a063790ac7c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8130f02a99048789abe30df533fcad1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:00.582228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7b39a2c2-2871-49b5-86be-a063790ac7c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8130f02a99048789abe30df533fcad1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:00.583127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7b39a2c2-2871-49b5-86be-a063790ac7c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8130f02a99048789abe30df533fcad1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:00.603959 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1203/2426] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:48:00 2022] DELETE /identity/v3/users/a8130f02a99048789abe30df533fcad1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:48:00.893719 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8c02b-dde7-4df4-99b8-3c8a9a7de47a admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:00.895326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-5eb8c02b-dde7-4df4-99b8-3c8a9a7de47a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:00.910909 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8c02b-dde7-4df4-99b8-3c8a9a7de47a admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:00.974188 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ddcb78-eeb1-436b-943b-105ea8f98ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:00.974759 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddcb78-eeb1-436b-943b-105ea8f98ffb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:00.975044 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddcb78-eeb1-436b-943b-105ea8f98ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:00.975319 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddcb78-eeb1-436b-943b-105ea8f98ffb admin admin] PATH_INFO: `/v3/projects/56f01f575abc448e8b2f660add386da7` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:00.976472 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ddcb78-eeb1-436b-943b-105ea8f98ffb admin admin] RBAC: Authorizing `identity:delete_project(project_id=56f01f575abc448e8b2f660add386da7)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:00.978544 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ddcb78-eeb1-436b-943b-105ea8f98ffb admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:01.024569 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-f6ddcb78-eeb1-436b-943b-105ea8f98ffb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56f01f575abc448e8b2f660add386da7', 'request_id': 'req-f6ddcb78-eeb1-436b-943b-105ea8f98ffb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:01.031361 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-f6ddcb78-eeb1-436b-943b-105ea8f98ffb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56f01f575abc448e8b2f660add386da7', 'request_id': 'req-f6ddcb78-eeb1-436b-943b-105ea8f98ffb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:01.039310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-f6ddcb78-eeb1-436b-943b-105ea8f98ffb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56f01f575abc448e8b2f660add386da7', 'request_id': 'req-f6ddcb78-eeb1-436b-943b-105ea8f98ffb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:01.050740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-f6ddcb78-eeb1-436b-943b-105ea8f98ffb admin admin] The token cache is being invalidate because project 56f01f575abc448e8b2f660add386da7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:48:01.051029 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-f6ddcb78-eeb1-436b-943b-105ea8f98ffb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:01.051652 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-f6ddcb78-eeb1-436b-943b-105ea8f98ffb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:01.053272 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1224/2427] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:48:00 2022] DELETE /identity/v3/projects/56f01f575abc448e8b2f660add386da7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:01.058853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a338823-495b-4051-a70a-3a0175aca657 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:01.059825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a338823-495b-4051-a70a-3a0175aca657 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:01.059825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a338823-495b-4051-a70a-3a0175aca657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:01.078995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1a338823-495b-4051-a70a-3a0175aca657 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:01.080179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1a338823-495b-4051-a70a-3a0175aca657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:01.138371 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1204/2428] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:01.146411 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a9659-ac13-4c7d-904b-d9de0a616229 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:01.146700 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a9659-ac13-4c7d-904b-d9de0a616229 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:01.146981 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a9659-ac13-4c7d-904b-d9de0a616229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:01.167067 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-916a9659-ac13-4c7d-904b-d9de0a616229 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:01.168342 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-916a9659-ac13-4c7d-904b-d9de0a616229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:01.176186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1225/2429] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:01.180688 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a338823-495b-4051-a70a-3a0175aca657 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:01.190541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a338823-495b-4051-a70a-3a0175aca657 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:01.195056 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f691cad-196a-4a31-9d08-c6d166888379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:01.195664 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f691cad-196a-4a31-9d08-c6d166888379 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:01.195946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f691cad-196a-4a31-9d08-c6d166888379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:01.196223 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f691cad-196a-4a31-9d08-c6d166888379 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:01.196919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f691cad-196a-4a31-9d08-c6d166888379 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:48:01.197262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f691cad-196a-4a31-9d08-c6d166888379 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:01.199387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f691cad-196a-4a31-9d08-c6d166888379 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:01.211429 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1205/2430] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:48:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:01.217037 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a9659-ac13-4c7d-904b-d9de0a616229 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:01.226927 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a9659-ac13-4c7d-904b-d9de0a616229 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:01.231804 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd5a9a-a351-42e5-9ef3-de31b27aa313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:01.232425 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd5a9a-a351-42e5-9ef3-de31b27aa313 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:01.232714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd5a9a-a351-42e5-9ef3-de31b27aa313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:01.233000 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd5a9a-a351-42e5-9ef3-de31b27aa313 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:01.233710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd5a9a-a351-42e5-9ef3-de31b27aa313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:01.235632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd5a9a-a351-42e5-9ef3-de31b27aa313 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:01.270861 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1226/2431] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:48:01 2022] POST /identity/v3/projects => generated 368 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:01.275228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f691cad-196a-4a31-9d08-c6d166888379 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:01.287699 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f691cad-196a-4a31-9d08-c6d166888379 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:01.352442 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe6819-6ae6-4b33-910d-779e822421aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:01.353075 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe6819-6ae6-4b33-910d-779e822421aa admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:01.353394 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe6819-6ae6-4b33-910d-779e822421aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:01.353678 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe6819-6ae6-4b33-910d-779e822421aa admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:01.354437 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe6819-6ae6-4b33-910d-779e822421aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:01.356622 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe6819-6ae6-4b33-910d-779e822421aa admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:01.375932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:01.383315 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1206/2432] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:48:01 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:48:01.387572 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd5a9a-a351-42e5-9ef3-de31b27aa313 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:01.398189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd5a9a-a351-42e5-9ef3-de31b27aa313 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:01.404224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16f903-9a60-481b-a629-e6da3bae5c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:01.405008 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16f903-9a60-481b-a629-e6da3bae5c99 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:01.405329 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16f903-9a60-481b-a629-e6da3bae5c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:01.405610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16f903-9a60-481b-a629-e6da3bae5c99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:01.406312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16f903-9a60-481b-a629-e6da3bae5c99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:01.407927 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16f903-9a60-481b-a629-e6da3bae5c99 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:01.410324 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:01.423959 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1227/2433] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:01 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:01.426005 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe6819-6ae6-4b33-910d-779e822421aa admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:01.438423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe6819-6ae6-4b33-910d-779e822421aa admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:01.443422 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c356e-bcf7-4163-a1c8-70efd46711bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:01.444054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c356e-bcf7-4163-a1c8-70efd46711bc admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:01.444333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c356e-bcf7-4163-a1c8-70efd46711bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:01.444604 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c356e-bcf7-4163-a1c8-70efd46711bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:01.445480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c356e-bcf7-4163-a1c8-70efd46711bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:01.447293 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c356e-bcf7-4163-a1c8-70efd46711bc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:01.460071 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1207/2434] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:01 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:01.465093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16f903-9a60-481b-a629-e6da3bae5c99 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:01.475549 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16f903-9a60-481b-a629-e6da3bae5c99 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:01.481375 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbd04f-6f58-4b65-ab11-6ad54b78fa55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:01.482061 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbd04f-6f58-4b65-ab11-6ad54b78fa55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:01.482356 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbd04f-6f58-4b65-ab11-6ad54b78fa55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:01.482631 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbd04f-6f58-4b65-ab11-6ad54b78fa55 admin admin] PATH_INFO: `/v3/projects/3706fd73bfcc4fc59753463fed59be99/users/a3d6ff2a17804d57bc674dd2ca6ca162/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:01.501498 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbd04f-6f58-4b65-ab11-6ad54b78fa55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3706fd73bfcc4fc59753463fed59be99, user_id=a3d6ff2a17804d57bc674dd2ca6ca162, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:01.504220 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbd04f-6f58-4b65-ab11-6ad54b78fa55 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:01.520679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1228/2435] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:01 2022] PUT /identity/v3/projects/3706fd73bfcc4fc59753463fed59be99/users/a3d6ff2a17804d57bc674dd2ca6ca162/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:01.532066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4aa81e-2c65-4509-867c-24cff2b01013 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:01.532356 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4aa81e-2c65-4509-867c-24cff2b01013 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:01.532639 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4aa81e-2c65-4509-867c-24cff2b01013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:01.562270 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9b4aa81e-2c65-4509-867c-24cff2b01013 None None] MFA Rules not processed for user `a3d6ff2a17804d57bc674dd2ca6ca162`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:01.563503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9b4aa81e-2c65-4509-867c-24cff2b01013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:01.646116 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1208/2436] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:01.650699 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbd04f-6f58-4b65-ab11-6ad54b78fa55 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:01.719411 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbd04f-6f58-4b65-ab11-6ad54b78fa55 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:01.724505 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f61918-3044-42a1-a9be-396038270d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:01.725310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f61918-3044-42a1-a9be-396038270d9c admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:01.725600 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f61918-3044-42a1-a9be-396038270d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:01.725876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f61918-3044-42a1-a9be-396038270d9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:01.726561 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f61918-3044-42a1-a9be-396038270d9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:01.728478 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f61918-3044-42a1-a9be-396038270d9c admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:01.758339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1229/2437] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:48:01 2022] POST /identity/v3/projects => generated 370 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:01.765010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4aa81e-2c65-4509-867c-24cff2b01013 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:01.776845 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4aa81e-2c65-4509-867c-24cff2b01013 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:01.847174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa4e8e-3485-4b6a-909e-c4090e2e9f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:01.847802 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa4e8e-3485-4b6a-909e-c4090e2e9f7a admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:01.848085 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa4e8e-3485-4b6a-909e-c4090e2e9f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:01.848357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa4e8e-3485-4b6a-909e-c4090e2e9f7a admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:01.849034 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa4e8e-3485-4b6a-909e-c4090e2e9f7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:01.851109 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa4e8e-3485-4b6a-909e-c4090e2e9f7a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:01.873448 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1209/2438] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:48:01 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:48:01.877581 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f61918-3044-42a1-a9be-396038270d9c admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:01.889197 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f61918-3044-42a1-a9be-396038270d9c admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:01.903071 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7b2c1-016d-4a3d-9ff7-17a37214a969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:01.904667 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7b2c1-016d-4a3d-9ff7-17a37214a969 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:01.904964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7b2c1-016d-4a3d-9ff7-17a37214a969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:01.905287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7b2c1-016d-4a3d-9ff7-17a37214a969 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:01.906017 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f7b2c1-016d-4a3d-9ff7-17a37214a969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:01.907669 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f7b2c1-016d-4a3d-9ff7-17a37214a969 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:01.921760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1230/2439] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:01 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:01.925789 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa4e8e-3485-4b6a-909e-c4090e2e9f7a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:01.938284 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa4e8e-3485-4b6a-909e-c4090e2e9f7a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:01.943203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed8dee-e90e-4e86-978e-a28d567a4fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:01.944014 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed8dee-e90e-4e86-978e-a28d567a4fc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:01.944330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed8dee-e90e-4e86-978e-a28d567a4fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:01.944609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed8dee-e90e-4e86-978e-a28d567a4fc9 admin admin] PATH_INFO: `/v3/projects/20fe0b0923cc43db8f40ee646757da08/users/f9d575affab1499dae8e99edb3b9533a/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:01.960068 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed8dee-e90e-4e86-978e-a28d567a4fc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20fe0b0923cc43db8f40ee646757da08, user_id=f9d575affab1499dae8e99edb3b9533a, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:01.962748 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed8dee-e90e-4e86-978e-a28d567a4fc9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:01.976490 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1210/2440] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:01 2022] PUT /identity/v3/projects/20fe0b0923cc43db8f40ee646757da08/users/f9d575affab1499dae8e99edb3b9533a/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:48:01.981094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7b2c1-016d-4a3d-9ff7-17a37214a969 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:01.991696 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7b2c1-016d-4a3d-9ff7-17a37214a969 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:02.051061 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6568c8-b8f7-4a74-8952-2ced6b065b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:02.051693 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6568c8-b8f7-4a74-8952-2ced6b065b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:02.052005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6568c8-b8f7-4a74-8952-2ced6b065b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:02.052282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6568c8-b8f7-4a74-8952-2ced6b065b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:02.052991 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6568c8-b8f7-4a74-8952-2ced6b065b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:02.054625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6568c8-b8f7-4a74-8952-2ced6b065b13 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:02.066009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1231/2441] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:01 2022] GET /identity/v3/roles => generated 1427 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:02.070095 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed8dee-e90e-4e86-978e-a28d567a4fc9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:02.080821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed8dee-e90e-4e86-978e-a28d567a4fc9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:02.085442 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc829b2b-3159-40d5-9ea2-2eff29ae06e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:02.086119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc829b2b-3159-40d5-9ea2-2eff29ae06e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:02.086402 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc829b2b-3159-40d5-9ea2-2eff29ae06e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:02.086679 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc829b2b-3159-40d5-9ea2-2eff29ae06e5 admin admin] PATH_INFO: `/v3/projects/20fe0b0923cc43db8f40ee646757da08/users/f9d575affab1499dae8e99edb3b9533a/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:02.088964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc829b2b-3159-40d5-9ea2-2eff29ae06e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20fe0b0923cc43db8f40ee646757da08, user_id=f9d575affab1499dae8e99edb3b9533a, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:02.091837 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc829b2b-3159-40d5-9ea2-2eff29ae06e5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:02.102594 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1211/2442] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:02 2022] PUT /identity/v3/projects/20fe0b0923cc43db8f40ee646757da08/users/f9d575affab1499dae8e99edb3b9533a/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:02.114176 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddc2fc-bd3d-4729-85e6-8a299748e9ca None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:02.114472 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddc2fc-bd3d-4729-85e6-8a299748e9ca None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:02.114757 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddc2fc-bd3d-4729-85e6-8a299748e9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:02.144025 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-ceddc2fc-bd3d-4729-85e6-8a299748e9ca None None] MFA Rules not processed for user `f9d575affab1499dae8e99edb3b9533a`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:02.145296 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ceddc2fc-bd3d-4729-85e6-8a299748e9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:02.181029 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:02.220184 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1232/2443] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:02 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:48:02.231835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc829b2b-3159-40d5-9ea2-2eff29ae06e5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:02.233629 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-dc829b2b-3159-40d5-9ea2-2eff29ae06e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:02.266989 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:02.295720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc829b2b-3159-40d5-9ea2-2eff29ae06e5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:02.300000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1d10b-5145-46d4-9657-4096e8ca631d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:02.300671 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1d10b-5145-46d4-9657-4096e8ca631d service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:02.300955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1d10b-5145-46d4-9657-4096e8ca631d service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:02.301279 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1d10b-5145-46d4-9657-4096e8ca631d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:02.311578 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1d10b-5145-46d4-9657-4096e8ca631d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:02.313022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1d10b-5145-46d4-9657-4096e8ca631d service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:02.323260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1212/2444] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:02 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:48:03.774841 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceddc2fc-bd3d-4729-85e6-8a299748e9ca None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:03.844809 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceddc2fc-bd3d-4729-85e6-8a299748e9ca None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:03.850499 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f902b-c767-40ad-9238-33d009280c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:03.851141 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f902b-c767-40ad-9238-33d009280c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:03.851418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f902b-c767-40ad-9238-33d009280c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:03.851689 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f902b-c767-40ad-9238-33d009280c16 admin admin] PATH_INFO: `/v3/users/a3d6ff2a17804d57bc674dd2ca6ca162` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:03.852855 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f902b-c767-40ad-9238-33d009280c16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3d6ff2a17804d57bc674dd2ca6ca162)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:03.854846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f902b-c767-40ad-9238-33d009280c16 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:03.924189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ad3f902b-c767-40ad-9238-33d009280c16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3d6ff2a17804d57bc674dd2ca6ca162'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:03.924924 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ad3f902b-c767-40ad-9238-33d009280c16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3d6ff2a17804d57bc674dd2ca6ca162'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:03.934474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ad3f902b-c767-40ad-9238-33d009280c16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3d6ff2a17804d57bc674dd2ca6ca162'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:03.946987 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ad3f902b-c767-40ad-9238-33d009280c16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3d6ff2a17804d57bc674dd2ca6ca162'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:03.954625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ad3f902b-c767-40ad-9238-33d009280c16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3d6ff2a17804d57bc674dd2ca6ca162'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:03.962737 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1233/2445] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:48:03 2022] DELETE /identity/v3/users/a3d6ff2a17804d57bc674dd2ca6ca162 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:03.967358 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2db452-a40b-41fb-8a31-a3fb4e269379 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:03.967870 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2db452-a40b-41fb-8a31-a3fb4e269379 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:03.967975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2db452-a40b-41fb-8a31-a3fb4e269379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:03.987896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-fe2db452-a40b-41fb-8a31-a3fb4e269379 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:03.989137 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fe2db452-a40b-41fb-8a31-a3fb4e269379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:04.055318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1213/2446] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:03 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:04.066147 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f902b-c767-40ad-9238-33d009280c16 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:04.067610 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ad3f902b-c767-40ad-9238-33d009280c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:04.128695 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:04.137507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f902b-c767-40ad-9238-33d009280c16 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:04.144558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e06696-87e7-4299-9c53-63a2f6997d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:04.144558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e06696-87e7-4299-9c53-63a2f6997d66 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:04.144558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e06696-87e7-4299-9c53-63a2f6997d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:04.144558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e06696-87e7-4299-9c53-63a2f6997d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:04.155180 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e06696-87e7-4299-9c53-63a2f6997d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:04.156624 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e06696-87e7-4299-9c53-63a2f6997d66 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:04.168408 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1234/2447] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:04 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:48:04.485283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2db452-a40b-41fb-8a31-a3fb4e269379 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:04.487128 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fe2db452-a40b-41fb-8a31-a3fb4e269379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:04.501251 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2db452-a40b-41fb-8a31-a3fb4e269379 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:04.510627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a3daa-6bf4-477e-bbdd-f5b86f731806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:04.511409 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a3daa-6bf4-477e-bbdd-f5b86f731806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:04.511703 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a3daa-6bf4-477e-bbdd-f5b86f731806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:04.511978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a3daa-6bf4-477e-bbdd-f5b86f731806 admin admin] PATH_INFO: `/v3/projects/3706fd73bfcc4fc59753463fed59be99` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:04.513342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a3daa-6bf4-477e-bbdd-f5b86f731806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3706fd73bfcc4fc59753463fed59be99)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:04.515467 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a3daa-6bf4-477e-bbdd-f5b86f731806 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:04.570603 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6d0a3daa-6bf4-477e-bbdd-f5b86f731806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3706fd73bfcc4fc59753463fed59be99', 'request_id': 'req-6d0a3daa-6bf4-477e-bbdd-f5b86f731806'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:04.577329 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6d0a3daa-6bf4-477e-bbdd-f5b86f731806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3706fd73bfcc4fc59753463fed59be99', 'request_id': 'req-6d0a3daa-6bf4-477e-bbdd-f5b86f731806'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:04.585921 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6d0a3daa-6bf4-477e-bbdd-f5b86f731806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3706fd73bfcc4fc59753463fed59be99', 'request_id': 'req-6d0a3daa-6bf4-477e-bbdd-f5b86f731806'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:04.597468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6d0a3daa-6bf4-477e-bbdd-f5b86f731806 admin admin] The token cache is being invalidate because project 3706fd73bfcc4fc59753463fed59be99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:48:04.597760 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6d0a3daa-6bf4-477e-bbdd-f5b86f731806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:04.598466 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6d0a3daa-6bf4-477e-bbdd-f5b86f731806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:04.599936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1214/2448] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:48:04 2022] DELETE /identity/v3/projects/3706fd73bfcc4fc59753463fed59be99 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:04.603921 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e06696-87e7-4299-9c53-63a2f6997d66 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:04.615124 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e06696-87e7-4299-9c53-63a2f6997d66 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:04.680423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb5b05-ffb8-43c7-a12c-002a66c9f8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:04.681077 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb5b05-ffb8-43c7-a12c-002a66c9f8a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:04.681407 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb5b05-ffb8-43c7-a12c-002a66c9f8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:04.681898 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb5b05-ffb8-43c7-a12c-002a66c9f8a5 admin admin] PATH_INFO: `/v3/users/f9d575affab1499dae8e99edb3b9533a` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:04.682939 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb5b05-ffb8-43c7-a12c-002a66c9f8a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9d575affab1499dae8e99edb3b9533a)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:04.684867 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb5b05-ffb8-43c7-a12c-002a66c9f8a5 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:04.745886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b5bb5b05-ffb8-43c7-a12c-002a66c9f8a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9d575affab1499dae8e99edb3b9533a'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:04.746850 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b5bb5b05-ffb8-43c7-a12c-002a66c9f8a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9d575affab1499dae8e99edb3b9533a'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:04.759735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b5bb5b05-ffb8-43c7-a12c-002a66c9f8a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9d575affab1499dae8e99edb3b9533a'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:04.772144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b5bb5b05-ffb8-43c7-a12c-002a66c9f8a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9d575affab1499dae8e99edb3b9533a'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:04.780356 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b5bb5b05-ffb8-43c7-a12c-002a66c9f8a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9d575affab1499dae8e99edb3b9533a'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:04.786878 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1235/2449] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:48:04 2022] DELETE /identity/v3/users/f9d575affab1499dae8e99edb3b9533a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:05.095822 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a3daa-6bf4-477e-bbdd-f5b86f731806 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:05.097591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6d0a3daa-6bf4-477e-bbdd-f5b86f731806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:05.114232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a3daa-6bf4-477e-bbdd-f5b86f731806 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:05.172658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:05.180738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1384336d-91c2-48fa-87ad-fad5faab8aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:05.181659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384336d-91c2-48fa-87ad-fad5faab8aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:05.182120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384336d-91c2-48fa-87ad-fad5faab8aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:05.182701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384336d-91c2-48fa-87ad-fad5faab8aa1 admin admin] PATH_INFO: `/v3/projects/20fe0b0923cc43db8f40ee646757da08` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:05.184094 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1384336d-91c2-48fa-87ad-fad5faab8aa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20fe0b0923cc43db8f40ee646757da08)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:05.186168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1384336d-91c2-48fa-87ad-fad5faab8aa1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:05.235135 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1384336d-91c2-48fa-87ad-fad5faab8aa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20fe0b0923cc43db8f40ee646757da08', 'request_id': 'req-1384336d-91c2-48fa-87ad-fad5faab8aa1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:05.241385 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1384336d-91c2-48fa-87ad-fad5faab8aa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20fe0b0923cc43db8f40ee646757da08', 'request_id': 'req-1384336d-91c2-48fa-87ad-fad5faab8aa1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:05.250783 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1384336d-91c2-48fa-87ad-fad5faab8aa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20fe0b0923cc43db8f40ee646757da08', 'request_id': 'req-1384336d-91c2-48fa-87ad-fad5faab8aa1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:05.264265 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1384336d-91c2-48fa-87ad-fad5faab8aa1 admin admin] The token cache is being invalidate because project 20fe0b0923cc43db8f40ee646757da08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:48:05.264557 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1384336d-91c2-48fa-87ad-fad5faab8aa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:05.265306 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1384336d-91c2-48fa-87ad-fad5faab8aa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:05.266867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1215/2450] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:48:05 2022] DELETE /identity/v3/projects/20fe0b0923cc43db8f40ee646757da08 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:05.271937 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94dc3b-2002-42bc-a733-66858eb2b65d None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:05.272243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94dc3b-2002-42bc-a733-66858eb2b65d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:05.272511 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94dc3b-2002-42bc-a733-66858eb2b65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:05.293703 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-1a94dc3b-2002-42bc-a733-66858eb2b65d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:05.294930 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1a94dc3b-2002-42bc-a733-66858eb2b65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:05.356175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1236/2451] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:05.364259 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1ffa7-4f6f-419e-b338-fe9aef133d36 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:05.364550 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1ffa7-4f6f-419e-b338-fe9aef133d36 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:05.364832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1ffa7-4f6f-419e-b338-fe9aef133d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:05.386923 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-89c1ffa7-4f6f-419e-b338-fe9aef133d36 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:05.388168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-89c1ffa7-4f6f-419e-b338-fe9aef133d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:05.396416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1216/2452] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:05.402042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94dc3b-2002-42bc-a733-66858eb2b65d None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:05.411909 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94dc3b-2002-42bc-a733-66858eb2b65d None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:05.416542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f978651-e4cb-4cbf-ae6f-e119b4b238fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:05.417388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f978651-e4cb-4cbf-ae6f-e119b4b238fc admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:05.417679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f978651-e4cb-4cbf-ae6f-e119b4b238fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:05.417956 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f978651-e4cb-4cbf-ae6f-e119b4b238fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:05.418684 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f978651-e4cb-4cbf-ae6f-e119b4b238fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:48:05.418982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f978651-e4cb-4cbf-ae6f-e119b4b238fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:05.420588 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f978651-e4cb-4cbf-ae6f-e119b4b238fc admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:05.428345 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:05.435080 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1237/2453] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:48:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:05.441715 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1ffa7-4f6f-419e-b338-fe9aef133d36 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:05.452432 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1ffa7-4f6f-419e-b338-fe9aef133d36 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:05.462210 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477cd52-c090-44d3-a0e1-7f756b761274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:05.462913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477cd52-c090-44d3-a0e1-7f756b761274 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:05.463195 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477cd52-c090-44d3-a0e1-7f756b761274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:05.463476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477cd52-c090-44d3-a0e1-7f756b761274 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:05.464207 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477cd52-c090-44d3-a0e1-7f756b761274 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:05.466189 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477cd52-c090-44d3-a0e1-7f756b761274 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:05.500896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1217/2454] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:48:05 2022] POST /identity/v3/projects => generated 372 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:05.505364 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f978651-e4cb-4cbf-ae6f-e119b4b238fc admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:05.517278 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f978651-e4cb-4cbf-ae6f-e119b4b238fc admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:05.577737 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c8363-8063-48c7-a741-f3cda7b2b830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:05.578366 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c8363-8063-48c7-a741-f3cda7b2b830 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:05.578654 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c8363-8063-48c7-a741-f3cda7b2b830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:05.578925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c8363-8063-48c7-a741-f3cda7b2b830 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:05.579591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c8363-8063-48c7-a741-f3cda7b2b830 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:05.581540 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c8363-8063-48c7-a741-f3cda7b2b830 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:05.605140 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1238/2455] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:48:05 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:05.609166 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477cd52-c090-44d3-a0e1-7f756b761274 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:05.624348 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477cd52-c090-44d3-a0e1-7f756b761274 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:05.629608 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe5221-db7b-478b-88b4-6da352b47186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:05.630255 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe5221-db7b-478b-88b4-6da352b47186 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:05.630534 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe5221-db7b-478b-88b4-6da352b47186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:05.631080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe5221-db7b-478b-88b4-6da352b47186 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:05.631805 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe5221-db7b-478b-88b4-6da352b47186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:05.633460 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe5221-db7b-478b-88b4-6da352b47186 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:05.649026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1218/2456] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:05 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:05.653224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c8363-8063-48c7-a741-f3cda7b2b830 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:05.664246 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c8363-8063-48c7-a741-f3cda7b2b830 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:05.674400 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee906133-4bb9-4b2e-b619-79ed4aae1c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:05.675260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee906133-4bb9-4b2e-b619-79ed4aae1c82 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:05.675500 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee906133-4bb9-4b2e-b619-79ed4aae1c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:05.675948 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee906133-4bb9-4b2e-b619-79ed4aae1c82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:05.676659 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee906133-4bb9-4b2e-b619-79ed4aae1c82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:05.678537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee906133-4bb9-4b2e-b619-79ed4aae1c82 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:05.690730 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1239/2457] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:05 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:05.695155 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe5221-db7b-478b-88b4-6da352b47186 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:05.706733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe5221-db7b-478b-88b4-6da352b47186 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:05.712576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d3ac6-30a2-44c5-9fd4-c8387e927f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:05.714401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d3ac6-30a2-44c5-9fd4-c8387e927f59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:05.714401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d3ac6-30a2-44c5-9fd4-c8387e927f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:05.714401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d3ac6-30a2-44c5-9fd4-c8387e927f59 admin admin] PATH_INFO: `/v3/projects/f99eef8b6cc64ee8aae548862078c315/users/4c2cd9bacdff43fdadf0759ef8f7abcc/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:05.735941 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d3ac6-30a2-44c5-9fd4-c8387e927f59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f99eef8b6cc64ee8aae548862078c315, user_id=4c2cd9bacdff43fdadf0759ef8f7abcc, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:05.739171 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d3ac6-30a2-44c5-9fd4-c8387e927f59 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:05.761142 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1219/2458] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:05 2022] PUT /identity/v3/projects/f99eef8b6cc64ee8aae548862078c315/users/4c2cd9bacdff43fdadf0759ef8f7abcc/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:48:05.762158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad46d8-d9f7-4f64-b981-09280d6dde3d None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:05.762451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad46d8-d9f7-4f64-b981-09280d6dde3d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:05.762732 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad46d8-d9f7-4f64-b981-09280d6dde3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:05.785492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-fdad46d8-d9f7-4f64-b981-09280d6dde3d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:05.786066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-fdad46d8-d9f7-4f64-b981-09280d6dde3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:05.867871 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1240/2459] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:05.878876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d3ac6-30a2-44c5-9fd4-c8387e927f59 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:05.881011 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-003d3ac6-30a2-44c5-9fd4-c8387e927f59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:05.923000 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad46d8-d9f7-4f64-b981-09280d6dde3d None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:05.924638 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-fdad46d8-d9f7-4f64-b981-09280d6dde3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:05.938335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad46d8-d9f7-4f64-b981-09280d6dde3d None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:05.943259 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446eb478-32a2-4286-823f-1fbeba3ff656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:05.943904 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446eb478-32a2-4286-823f-1fbeba3ff656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:05.944186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446eb478-32a2-4286-823f-1fbeba3ff656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:05.944460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446eb478-32a2-4286-823f-1fbeba3ff656 admin admin] PATH_INFO: `/v3/users/fa6a3343bb7b4e9f9327bbcbebea5ebe` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:05.945668 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446eb478-32a2-4286-823f-1fbeba3ff656 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa6a3343bb7b4e9f9327bbcbebea5ebe)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:05.947603 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446eb478-32a2-4286-823f-1fbeba3ff656 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:05.955930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d3ac6-30a2-44c5-9fd4-c8387e927f59 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:05.959947 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125cdbf-5f0f-432b-ad11-27903153cac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:05.960577 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125cdbf-5f0f-432b-ad11-27903153cac3 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:05.960865 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125cdbf-5f0f-432b-ad11-27903153cac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:05.961172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125cdbf-5f0f-432b-ad11-27903153cac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:05.971565 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125cdbf-5f0f-432b-ad11-27903153cac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:05.973177 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125cdbf-5f0f-432b-ad11-27903153cac3 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:05.984680 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1220/2460] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:05 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:48:06.013703 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-446eb478-32a2-4286-823f-1fbeba3ff656 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa6a3343bb7b4e9f9327bbcbebea5ebe'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:06.014587 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-446eb478-32a2-4286-823f-1fbeba3ff656 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa6a3343bb7b4e9f9327bbcbebea5ebe'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:06.024982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-446eb478-32a2-4286-823f-1fbeba3ff656 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa6a3343bb7b4e9f9327bbcbebea5ebe'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:06.039609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-446eb478-32a2-4286-823f-1fbeba3ff656 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa6a3343bb7b4e9f9327bbcbebea5ebe'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:06.041427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:06.051515 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-446eb478-32a2-4286-823f-1fbeba3ff656 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa6a3343bb7b4e9f9327bbcbebea5ebe'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:06.060231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1241/2461] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:48:05 2022] DELETE /identity/v3/users/fa6a3343bb7b4e9f9327bbcbebea5ebe => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:48:06.065861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e01659-9519-4824-912c-4bd0894dcb5b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:06.066365 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e01659-9519-4824-912c-4bd0894dcb5b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:06.066782 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e01659-9519-4824-912c-4bd0894dcb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:06.089225 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-63e01659-9519-4824-912c-4bd0894dcb5b None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:06.090681 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-63e01659-9519-4824-912c-4bd0894dcb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:06.114619 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:06.165526 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1221/2462] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:48:06.175468 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446eb478-32a2-4286-823f-1fbeba3ff656 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:06.177315 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-446eb478-32a2-4286-823f-1fbeba3ff656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:06.384131 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446eb478-32a2-4286-823f-1fbeba3ff656 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:06.390722 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ac096-c879-4c58-90a0-c4b0351e2ea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:06.390722 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ac096-c879-4c58-90a0-c4b0351e2ea8 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:06.390722 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ac096-c879-4c58-90a0-c4b0351e2ea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:06.390722 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ac096-c879-4c58-90a0-c4b0351e2ea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:06.403740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ac096-c879-4c58-90a0-c4b0351e2ea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:06.405344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ac096-c879-4c58-90a0-c4b0351e2ea8 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:06.420474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1242/2463] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:06 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:48:06.862567 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad2ee1-30b0-4d72-aa6e-1547bbeaf2fa None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:06.862863 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad2ee1-30b0-4d72-aa6e-1547bbeaf2fa None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:06.863672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad2ee1-30b0-4d72-aa6e-1547bbeaf2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:06.897793 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a0ad2ee1-30b0-4d72-aa6e-1547bbeaf2fa None None] MFA Rules not processed for user `4c2cd9bacdff43fdadf0759ef8f7abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:06.899420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a0ad2ee1-30b0-4d72-aa6e-1547bbeaf2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:06.914278 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ac096-c879-4c58-90a0-c4b0351e2ea8 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:06.916381 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-eb5ac096-c879-4c58-90a0-c4b0351e2ea8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:06.930280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ac096-c879-4c58-90a0-c4b0351e2ea8 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:06.936737 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fd27f-6b39-41ad-a6db-20050ac34132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:06.937406 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fd27f-6b39-41ad-a6db-20050ac34132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:06.937717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fd27f-6b39-41ad-a6db-20050ac34132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:06.937965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fd27f-6b39-41ad-a6db-20050ac34132 admin admin] PATH_INFO: `/v3/projects/fec27884e3dc476bb8d0c46d0b33c1d9` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:06.939187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fd27f-6b39-41ad-a6db-20050ac34132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fec27884e3dc476bb8d0c46d0b33c1d9)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:06.941481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fd27f-6b39-41ad-a6db-20050ac34132 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:07.001954 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-f81fd27f-6b39-41ad-a6db-20050ac34132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fec27884e3dc476bb8d0c46d0b33c1d9', 'request_id': 'req-f81fd27f-6b39-41ad-a6db-20050ac34132'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:07.009673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-f81fd27f-6b39-41ad-a6db-20050ac34132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fec27884e3dc476bb8d0c46d0b33c1d9', 'request_id': 'req-f81fd27f-6b39-41ad-a6db-20050ac34132'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:07.013724 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1222/2464] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:07.018285 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad2ee1-30b0-4d72-aa6e-1547bbeaf2fa None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:07.019948 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a0ad2ee1-30b0-4d72-aa6e-1547bbeaf2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:07.025327 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-f81fd27f-6b39-41ad-a6db-20050ac34132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fec27884e3dc476bb8d0c46d0b33c1d9', 'request_id': 'req-f81fd27f-6b39-41ad-a6db-20050ac34132'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:07.035658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad2ee1-30b0-4d72-aa6e-1547bbeaf2fa None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:07.039001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-f81fd27f-6b39-41ad-a6db-20050ac34132 admin admin] The token cache is being invalidate because project fec27884e3dc476bb8d0c46d0b33c1d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:48:07.039519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-f81fd27f-6b39-41ad-a6db-20050ac34132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:07.040174 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-f81fd27f-6b39-41ad-a6db-20050ac34132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:07.045178 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1243/2465] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:48:06 2022] DELETE /identity/v3/projects/fec27884e3dc476bb8d0c46d0b33c1d9 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:48:07.045988 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fd27f-6b39-41ad-a6db-20050ac34132 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:07.048150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f81fd27f-6b39-41ad-a6db-20050ac34132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:07.112774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28745ca9-e20f-4f26-9d2f-59dd41fcb039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:07.113457 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28745ca9-e20f-4f26-9d2f-59dd41fcb039 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:07.113979 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28745ca9-e20f-4f26-9d2f-59dd41fcb039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:07.114504 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28745ca9-e20f-4f26-9d2f-59dd41fcb039 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:07.115014 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28745ca9-e20f-4f26-9d2f-59dd41fcb039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:07.116943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28745ca9-e20f-4f26-9d2f-59dd41fcb039 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:07.141017 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fd27f-6b39-41ad-a6db-20050ac34132 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:07.146567 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54e0bf-4307-4112-b8a6-20e1836a7a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:07.147608 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54e0bf-4307-4112-b8a6-20e1836a7a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:07.147977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54e0bf-4307-4112-b8a6-20e1836a7a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:07.148405 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54e0bf-4307-4112-b8a6-20e1836a7a67 admin admin] PATH_INFO: `/v3/users/88b21ff05f2d467d84415c0bad4a0ae9` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:07.149958 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1223/2466] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:48:07 2022] POST /identity/v3/projects => generated 374 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:48:07.150201 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54e0bf-4307-4112-b8a6-20e1836a7a67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88b21ff05f2d467d84415c0bad4a0ae9)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:07.154053 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54e0bf-4307-4112-b8a6-20e1836a7a67 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:07.154859 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28745ca9-e20f-4f26-9d2f-59dd41fcb039 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:07.156892 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-28745ca9-e20f-4f26-9d2f-59dd41fcb039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:07.181090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:07.224166 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:07.234211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8e54e0bf-4307-4112-b8a6-20e1836a7a67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88b21ff05f2d467d84415c0bad4a0ae9'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:07.235051 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8e54e0bf-4307-4112-b8a6-20e1836a7a67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88b21ff05f2d467d84415c0bad4a0ae9'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:07.244003 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8e54e0bf-4307-4112-b8a6-20e1836a7a67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88b21ff05f2d467d84415c0bad4a0ae9'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:07.257326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8e54e0bf-4307-4112-b8a6-20e1836a7a67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88b21ff05f2d467d84415c0bad4a0ae9'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:07.264493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8e54e0bf-4307-4112-b8a6-20e1836a7a67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88b21ff05f2d467d84415c0bad4a0ae9'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:07.266916 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28745ca9-e20f-4f26-9d2f-59dd41fcb039 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:07.270953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1244/2467] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:48:07 2022] DELETE /identity/v3/users/88b21ff05f2d467d84415c0bad4a0ae9 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:48:07.274901 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02edd6-8e4c-4f20-84fd-f6e3879f0da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:07.275767 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02edd6-8e4c-4f20-84fd-f6e3879f0da1 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:07.276304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02edd6-8e4c-4f20-84fd-f6e3879f0da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:07.276780 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02edd6-8e4c-4f20-84fd-f6e3879f0da1 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:07.278017 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02edd6-8e4c-4f20-84fd-f6e3879f0da1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:07.281877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02edd6-8e4c-4f20-84fd-f6e3879f0da1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:07.308727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1224/2468] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:48:07 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:48:07.312800 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54e0bf-4307-4112-b8a6-20e1836a7a67 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:07.314241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-8e54e0bf-4307-4112-b8a6-20e1836a7a67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:07.397573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54e0bf-4307-4112-b8a6-20e1836a7a67 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:07.403248 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d1d22-f0be-4c98-8777-9dd68e86b03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:07.403889 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d1d22-f0be-4c98-8777-9dd68e86b03b admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:07.404164 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d1d22-f0be-4c98-8777-9dd68e86b03b admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:07.404429 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d1d22-f0be-4c98-8777-9dd68e86b03b admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:07.405150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d1d22-f0be-4c98-8777-9dd68e86b03b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:07.406768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d1d22-f0be-4c98-8777-9dd68e86b03b admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:07.419985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1245/2469] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:07 2022] GET /identity/v3/roles => generated 1427 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:07.424876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02edd6-8e4c-4f20-84fd-f6e3879f0da1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:07.436539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02edd6-8e4c-4f20-84fd-f6e3879f0da1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:07.443168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728941cd-dd09-4475-ad20-8db0be3794a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:07.443673 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728941cd-dd09-4475-ad20-8db0be3794a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:07.443955 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728941cd-dd09-4475-ad20-8db0be3794a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:07.444229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728941cd-dd09-4475-ad20-8db0be3794a7 admin admin] PATH_INFO: `/v3/projects/79614c93071c4352a501b289e4609ba4/users/0870d660b03745dc9cba85b0f729fdb6/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:07.464440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728941cd-dd09-4475-ad20-8db0be3794a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79614c93071c4352a501b289e4609ba4, user_id=0870d660b03745dc9cba85b0f729fdb6, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:07.469619 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728941cd-dd09-4475-ad20-8db0be3794a7 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:07.492623 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1225/2470] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:07 2022] PUT /identity/v3/projects/79614c93071c4352a501b289e4609ba4/users/0870d660b03745dc9cba85b0f729fdb6/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:07.497498 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d1d22-f0be-4c98-8777-9dd68e86b03b admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:07.509475 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d1d22-f0be-4c98-8777-9dd68e86b03b admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:07.575854 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2611287-d59c-48cf-8ac3-0330ae7fefa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:07.576488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2611287-d59c-48cf-8ac3-0330ae7fefa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:07.576770 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2611287-d59c-48cf-8ac3-0330ae7fefa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:07.577097 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2611287-d59c-48cf-8ac3-0330ae7fefa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:07.577824 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2611287-d59c-48cf-8ac3-0330ae7fefa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:07.579427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2611287-d59c-48cf-8ac3-0330ae7fefa3 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:07.590882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1246/2471] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:07 2022] GET /identity/v3/roles => generated 1427 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:07.595403 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728941cd-dd09-4475-ad20-8db0be3794a7 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:07.607597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728941cd-dd09-4475-ad20-8db0be3794a7 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:07.607904 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2611287-d59c-48cf-8ac3-0330ae7fefa3 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:07.609701 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c2611287-d59c-48cf-8ac3-0330ae7fefa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:07.613261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f08842-89c9-480a-b102-767caeed4aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:07.614222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f08842-89c9-480a-b102-767caeed4aaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:07.614515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f08842-89c9-480a-b102-767caeed4aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:07.614794 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f08842-89c9-480a-b102-767caeed4aaa admin admin] PATH_INFO: `/v3/projects/79614c93071c4352a501b289e4609ba4/users/0870d660b03745dc9cba85b0f729fdb6/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:07.617239 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f08842-89c9-480a-b102-767caeed4aaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=79614c93071c4352a501b289e4609ba4, user_id=0870d660b03745dc9cba85b0f729fdb6, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:07.620118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f08842-89c9-480a-b102-767caeed4aaa admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:07.627177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2611287-d59c-48cf-8ac3-0330ae7fefa3 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:07.635522 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1226/2472] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:07 2022] PUT /identity/v3/projects/79614c93071c4352a501b289e4609ba4/users/0870d660b03745dc9cba85b0f729fdb6/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:07.658056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:07.696146 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781cc8ac-4c4b-4bb1-b506-8f48d57cd0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:07.696908 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cc8ac-4c4b-4bb1-b506-8f48d57cd0d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:07.697210 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cc8ac-4c4b-4bb1-b506-8f48d57cd0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:07.697488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cc8ac-4c4b-4bb1-b506-8f48d57cd0d2 admin admin] PATH_INFO: `/v3/projects/5d49a05313be4b38b53d09efe9f7683a` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:07.698913 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781cc8ac-4c4b-4bb1-b506-8f48d57cd0d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d49a05313be4b38b53d09efe9f7683a)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:07.700835 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781cc8ac-4c4b-4bb1-b506-8f48d57cd0d2 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:07.754637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-781cc8ac-4c4b-4bb1-b506-8f48d57cd0d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d49a05313be4b38b53d09efe9f7683a', 'request_id': 'req-781cc8ac-4c4b-4bb1-b506-8f48d57cd0d2'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:07.762288 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-781cc8ac-4c4b-4bb1-b506-8f48d57cd0d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d49a05313be4b38b53d09efe9f7683a', 'request_id': 'req-781cc8ac-4c4b-4bb1-b506-8f48d57cd0d2'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:07.770642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-781cc8ac-4c4b-4bb1-b506-8f48d57cd0d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d49a05313be4b38b53d09efe9f7683a', 'request_id': 'req-781cc8ac-4c4b-4bb1-b506-8f48d57cd0d2'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:07.784219 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-781cc8ac-4c4b-4bb1-b506-8f48d57cd0d2 admin admin] The token cache is being invalidate because project 5d49a05313be4b38b53d09efe9f7683a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:48:07.784607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-781cc8ac-4c4b-4bb1-b506-8f48d57cd0d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:07.785434 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-781cc8ac-4c4b-4bb1-b506-8f48d57cd0d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:07.787021 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1247/2473] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:48:07 2022] DELETE /identity/v3/projects/5d49a05313be4b38b53d09efe9f7683a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:48:07.793249 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c06a4c-ba1e-4340-b8df-7639ecdba2bb None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:07.793641 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c06a4c-ba1e-4340-b8df-7639ecdba2bb None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:07.793997 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c06a4c-ba1e-4340-b8df-7639ecdba2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:07.817791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-89c06a4c-ba1e-4340-b8df-7639ecdba2bb None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:07.818943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-89c06a4c-ba1e-4340-b8df-7639ecdba2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:07.936248 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1227/2474] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:07.945735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c80a13-eb49-4adb-ba86-adfe50cc3b28 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:07.947568 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c80a13-eb49-4adb-ba86-adfe50cc3b28 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:07.948925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c80a13-eb49-4adb-ba86-adfe50cc3b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:07.981823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-f6c80a13-eb49-4adb-ba86-adfe50cc3b28 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:07.983155 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f6c80a13-eb49-4adb-ba86-adfe50cc3b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:07.994327 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1248/2475] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:08.000061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c06a4c-ba1e-4340-b8df-7639ecdba2bb None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:08.011416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c06a4c-ba1e-4340-b8df-7639ecdba2bb None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:08.022673 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc5c15-df1b-4bde-b10a-9c988b2a3c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:08.023343 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc5c15-df1b-4bde-b10a-9c988b2a3c87 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:08.023635 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc5c15-df1b-4bde-b10a-9c988b2a3c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:08.023936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc5c15-df1b-4bde-b10a-9c988b2a3c87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:08.024883 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc5c15-df1b-4bde-b10a-9c988b2a3c87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:48:08.025474 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc5c15-df1b-4bde-b10a-9c988b2a3c87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:08.029015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc5c15-df1b-4bde-b10a-9c988b2a3c87 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:08.046806 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1228/2476] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:48:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:08.051177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c80a13-eb49-4adb-ba86-adfe50cc3b28 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:08.062846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c80a13-eb49-4adb-ba86-adfe50cc3b28 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:08.067663 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f738883-36e7-458c-ba0b-7d78f7ea0ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:08.068296 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f738883-36e7-458c-ba0b-7d78f7ea0ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:08.068581 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f738883-36e7-458c-ba0b-7d78f7ea0ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:08.068859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f738883-36e7-458c-ba0b-7d78f7ea0ae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:08.069584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f738883-36e7-458c-ba0b-7d78f7ea0ae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:08.071532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f738883-36e7-458c-ba0b-7d78f7ea0ae7 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:08.106342 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1249/2477] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:48:08 2022] POST /identity/v3/projects => generated 388 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:08.112026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc5c15-df1b-4bde-b10a-9c988b2a3c87 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:08.125842 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc5c15-df1b-4bde-b10a-9c988b2a3c87 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:08.193254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43106172-b32b-4b03-9d37-fdbd4a8cc3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:08.193905 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43106172-b32b-4b03-9d37-fdbd4a8cc3ba admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:08.194226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43106172-b32b-4b03-9d37-fdbd4a8cc3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:08.194514 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43106172-b32b-4b03-9d37-fdbd4a8cc3ba admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:08.195268 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43106172-b32b-4b03-9d37-fdbd4a8cc3ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:08.197296 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43106172-b32b-4b03-9d37-fdbd4a8cc3ba admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:08.220004 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1229/2478] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:48:08 2022] POST /identity/v3/users => generated 295 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:08.225544 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f738883-36e7-458c-ba0b-7d78f7ea0ae7 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:08.236976 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f738883-36e7-458c-ba0b-7d78f7ea0ae7 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:08.241744 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cb0bd-cfda-47c0-a67c-23c952126cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:08.242360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cb0bd-cfda-47c0-a67c-23c952126cdd admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:08.242643 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cb0bd-cfda-47c0-a67c-23c952126cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:08.242917 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cb0bd-cfda-47c0-a67c-23c952126cdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:08.243619 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cb0bd-cfda-47c0-a67c-23c952126cdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:08.245828 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cb0bd-cfda-47c0-a67c-23c952126cdd admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:08.256835 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1250/2479] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:08 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:08.260914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43106172-b32b-4b03-9d37-fdbd4a8cc3ba admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:08.271707 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43106172-b32b-4b03-9d37-fdbd4a8cc3ba admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:08.277281 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676af9f-2034-4f13-bd58-7344f53e0b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:08.278019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676af9f-2034-4f13-bd58-7344f53e0b3f admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:08.278304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676af9f-2034-4f13-bd58-7344f53e0b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:08.278578 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676af9f-2034-4f13-bd58-7344f53e0b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:08.279392 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676af9f-2034-4f13-bd58-7344f53e0b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:08.281022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676af9f-2034-4f13-bd58-7344f53e0b3f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:08.283400 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:08.296704 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1230/2480] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:08 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:08.300806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cb0bd-cfda-47c0-a67c-23c952126cdd admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:08.311103 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cb0bd-cfda-47c0-a67c-23c952126cdd admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:08.315828 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63fcd43-434c-45be-a4f7-397a62230682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:08.316503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63fcd43-434c-45be-a4f7-397a62230682 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:08.316786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63fcd43-434c-45be-a4f7-397a62230682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:08.317063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63fcd43-434c-45be-a4f7-397a62230682 admin admin] PATH_INFO: `/v3/projects/1bb28576ea31403da246cbfffe90e7db/users/031ae4e996b145d2b71958cc8d902386/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:08.336317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63fcd43-434c-45be-a4f7-397a62230682 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bb28576ea31403da246cbfffe90e7db, user_id=031ae4e996b145d2b71958cc8d902386, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:08.341770 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63fcd43-434c-45be-a4f7-397a62230682 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:08.359801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1251/2481] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:08 2022] PUT /identity/v3/projects/1bb28576ea31403da246cbfffe90e7db/users/031ae4e996b145d2b71958cc8d902386/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:08.369778 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc3595-ebe8-4e28-8c35-123b98510211 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:08.369778 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc3595-ebe8-4e28-8c35-123b98510211 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:08.369778 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc3595-ebe8-4e28-8c35-123b98510211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:08.371778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f776bc-576b-4309-8eab-dd34d55c7c7a None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:08.372083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f776bc-576b-4309-8eab-dd34d55c7c7a None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:08.372357 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f776bc-576b-4309-8eab-dd34d55c7c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:08.402892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-b0f776bc-576b-4309-8eab-dd34d55c7c7a None None] MFA Rules not processed for user `031ae4e996b145d2b71958cc8d902386`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:08.404118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b0f776bc-576b-4309-8eab-dd34d55c7c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:08.405034 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-71cc3595-ebe8-4e28-8c35-123b98510211 None None] MFA Rules not processed for user `0870d660b03745dc9cba85b0f729fdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:08.406253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-71cc3595-ebe8-4e28-8c35-123b98510211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:08.462780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:08.494036 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1252/2482] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:08 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:48:08.499283 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f776bc-576b-4309-8eab-dd34d55c7c7a None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:08.508246 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1231/2483] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:08 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:08.521038 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc3595-ebe8-4e28-8c35-123b98510211 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:08.522598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-71cc3595-ebe8-4e28-8c35-123b98510211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:08.590261 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f776bc-576b-4309-8eab-dd34d55c7c7a None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:08.591242 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc3595-ebe8-4e28-8c35-123b98510211 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:08.594799 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd18e54-c855-42e3-871e-1a9446abdeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:08.594960 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ffc6e-66d6-4a4d-868e-ff9466cb7ce1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:08.595473 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd18e54-c855-42e3-871e-1a9446abdeb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:08.595594 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ffc6e-66d6-4a4d-868e-ff9466cb7ce1 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:08.595760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd18e54-c855-42e3-871e-1a9446abdeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:08.595888 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ffc6e-66d6-4a4d-868e-ff9466cb7ce1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:08.596036 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd18e54-c855-42e3-871e-1a9446abdeb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:08.596141 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ffc6e-66d6-4a4d-868e-ff9466cb7ce1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:08.596695 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd18e54-c855-42e3-871e-1a9446abdeb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:08.597856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bf5ffc6e-66d6-4a4d-868e-ff9466cb7ce1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:08.598641 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd18e54-c855-42e3-871e-1a9446abdeb2 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:08.628629 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1253/2484] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:48:08 2022] POST /identity/v3/projects => generated 386 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:08.633047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd18e54-c855-42e3-871e-1a9446abdeb2 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:08.663170 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ffc6e-66d6-4a4d-868e-ff9466cb7ce1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:08.666397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ffc6e-66d6-4a4d-868e-ff9466cb7ce1 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:08.675377 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1232/2485] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:48:08.702143 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd18e54-c855-42e3-871e-1a9446abdeb2 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:08.706915 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0d182-500c-45c5-97e9-332b9de60458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:08.707579 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0d182-500c-45c5-97e9-332b9de60458 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:08.707887 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0d182-500c-45c5-97e9-332b9de60458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:08.708168 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0d182-500c-45c5-97e9-332b9de60458 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:08.708887 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0d182-500c-45c5-97e9-332b9de60458 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:08.710978 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0d182-500c-45c5-97e9-332b9de60458 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:08.733589 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1254/2486] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:48:08 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:08.738447 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ffc6e-66d6-4a4d-868e-ff9466cb7ce1 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:08.752067 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ffc6e-66d6-4a4d-868e-ff9466cb7ce1 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:08.757419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6d072-65d1-41cd-816c-ddc66f3ff9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:08.758074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6d072-65d1-41cd-816c-ddc66f3ff9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:08.758355 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6d072-65d1-41cd-816c-ddc66f3ff9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:08.758631 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6d072-65d1-41cd-816c-ddc66f3ff9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:08.759323 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6d072-65d1-41cd-816c-ddc66f3ff9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:08.761154 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6d072-65d1-41cd-816c-ddc66f3ff9a6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:08.773158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1233/2487] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:08 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:08.777801 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0d182-500c-45c5-97e9-332b9de60458 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:08.788431 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0d182-500c-45c5-97e9-332b9de60458 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:08.794093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccad38-364e-42b9-8f31-817d230c5867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:08.794887 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccad38-364e-42b9-8f31-817d230c5867 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:08.795265 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccad38-364e-42b9-8f31-817d230c5867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:08.795547 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccad38-364e-42b9-8f31-817d230c5867 admin admin] PATH_INFO: `/v3/projects/d8f18c796d814a948be8754c4dc52f8f/users/dc9390b9dbc549fd86444f1a9fead9cf/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:08.813297 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ccad38-364e-42b9-8f31-817d230c5867 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8f18c796d814a948be8754c4dc52f8f, user_id=dc9390b9dbc549fd86444f1a9fead9cf, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:08.816013 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ccad38-364e-42b9-8f31-817d230c5867 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:08.830238 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1255/2488] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:08 2022] PUT /identity/v3/projects/d8f18c796d814a948be8754c4dc52f8f/users/dc9390b9dbc549fd86444f1a9fead9cf/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:08.834462 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6d072-65d1-41cd-816c-ddc66f3ff9a6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:08.845171 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6d072-65d1-41cd-816c-ddc66f3ff9a6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:08.875157 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:08.913828 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c28c4-c954-4607-8461-e6aa9c4627bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:08.914480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c28c4-c954-4607-8461-e6aa9c4627bc admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:08.914765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c28c4-c954-4607-8461-e6aa9c4627bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:08.915046 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c28c4-c954-4607-8461-e6aa9c4627bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:08.915875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c28c4-c954-4607-8461-e6aa9c4627bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:08.918332 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c28c4-c954-4607-8461-e6aa9c4627bc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:08.934238 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1234/2489] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:08 2022] GET /identity/v3/roles => generated 1427 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:48:08.938799 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccad38-364e-42b9-8f31-817d230c5867 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:08.951980 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccad38-364e-42b9-8f31-817d230c5867 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:08.959523 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97d129-483c-463d-a3ca-03663ba0708e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:08.960265 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97d129-483c-463d-a3ca-03663ba0708e admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:08.960591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97d129-483c-463d-a3ca-03663ba0708e admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:08.960889 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97d129-483c-463d-a3ca-03663ba0708e admin admin] PATH_INFO: `/v3/projects/d8f18c796d814a948be8754c4dc52f8f/users/dc9390b9dbc549fd86444f1a9fead9cf/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:08.964150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97d129-483c-463d-a3ca-03663ba0708e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8f18c796d814a948be8754c4dc52f8f, user_id=dc9390b9dbc549fd86444f1a9fead9cf, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:08.967974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97d129-483c-463d-a3ca-03663ba0708e admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:08.987458 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1256/2490] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:08 2022] PUT /identity/v3/projects/d8f18c796d814a948be8754c4dc52f8f/users/dc9390b9dbc549fd86444f1a9fead9cf/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:09.009801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019ef61-024b-48b4-a76e-b42ff0edb248 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:09.009801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019ef61-024b-48b4-a76e-b42ff0edb248 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:09.009801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019ef61-024b-48b4-a76e-b42ff0edb248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:09.057291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6019ef61-024b-48b4-a76e-b42ff0edb248 None None] MFA Rules not processed for user `dc9390b9dbc549fd86444f1a9fead9cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:09.058948 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6019ef61-024b-48b4-a76e-b42ff0edb248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:09.137766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1235/2491] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:08 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:09.148930 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97d129-483c-463d-a3ca-03663ba0708e admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:09.150766 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-7e97d129-483c-463d-a3ca-03663ba0708e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:09.183413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:09.216282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97d129-483c-463d-a3ca-03663ba0708e admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:09.220008 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78c535-f589-4f2e-a32d-5253493fcea2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:09.220625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78c535-f589-4f2e-a32d-5253493fcea2 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:09.220909 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78c535-f589-4f2e-a32d-5253493fcea2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:09.221218 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78c535-f589-4f2e-a32d-5253493fcea2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:09.231411 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78c535-f589-4f2e-a32d-5253493fcea2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:09.232865 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78c535-f589-4f2e-a32d-5253493fcea2 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:09.242644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1257/2492] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:09 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:48:09.271037 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019ef61-024b-48b4-a76e-b42ff0edb248 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:09.334092 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019ef61-024b-48b4-a76e-b42ff0edb248 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:09.339138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a661aa3-1028-4697-ad86-a937b84f3ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:09.339775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a661aa3-1028-4697-ad86-a937b84f3ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:09.340063 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a661aa3-1028-4697-ad86-a937b84f3ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:09.340345 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a661aa3-1028-4697-ad86-a937b84f3ad2 admin admin] PATH_INFO: `/v3/users/031ae4e996b145d2b71958cc8d902386` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:09.341547 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a661aa3-1028-4697-ad86-a937b84f3ad2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=031ae4e996b145d2b71958cc8d902386)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:09.343472 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a661aa3-1028-4697-ad86-a937b84f3ad2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:09.391169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0a661aa3-1028-4697-ad86-a937b84f3ad2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '031ae4e996b145d2b71958cc8d902386'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:09.393445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:09.399513 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0a661aa3-1028-4697-ad86-a937b84f3ad2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '031ae4e996b145d2b71958cc8d902386'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:09.400345 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0a661aa3-1028-4697-ad86-a937b84f3ad2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '031ae4e996b145d2b71958cc8d902386'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:09.408677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0a661aa3-1028-4697-ad86-a937b84f3ad2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '031ae4e996b145d2b71958cc8d902386'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:09.409379 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0a661aa3-1028-4697-ad86-a937b84f3ad2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '031ae4e996b145d2b71958cc8d902386'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:09.426979 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1236/2493] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:48:09 2022] DELETE /identity/v3/users/031ae4e996b145d2b71958cc8d902386 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:09.431574 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe2bf4-fa36-4fe8-8cd9-c2f5837fad18 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:09.431904 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe2bf4-fa36-4fe8-8cd9-c2f5837fad18 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:09.432197 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe2bf4-fa36-4fe8-8cd9-c2f5837fad18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:09.451441 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-5cfe2bf4-fa36-4fe8-8cd9-c2f5837fad18 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:09.452633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-5cfe2bf4-fa36-4fe8-8cd9-c2f5837fad18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:09.511052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1258/2494] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:09.521279 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a661aa3-1028-4697-ad86-a937b84f3ad2 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:09.522868 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0a661aa3-1028-4697-ad86-a937b84f3ad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:09.583712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a661aa3-1028-4697-ad86-a937b84f3ad2 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:09.587514 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0167a5-8deb-4578-b783-9245e90e1858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:09.588066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0167a5-8deb-4578-b783-9245e90e1858 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:09.588347 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0167a5-8deb-4578-b783-9245e90e1858 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:09.588739 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0167a5-8deb-4578-b783-9245e90e1858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:09.598169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0167a5-8deb-4578-b783-9245e90e1858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:09.599697 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0167a5-8deb-4578-b783-9245e90e1858 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:09.609946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1237/2495] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:09 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:48:09.911852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe2bf4-fa36-4fe8-8cd9-c2f5837fad18 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:09.913439 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-5cfe2bf4-fa36-4fe8-8cd9-c2f5837fad18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:09.928833 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe2bf4-fa36-4fe8-8cd9-c2f5837fad18 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:09.933964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455c0d6-5e24-4a4b-9a9a-601d8dc6af30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:09.934618 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455c0d6-5e24-4a4b-9a9a-601d8dc6af30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:09.934996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455c0d6-5e24-4a4b-9a9a-601d8dc6af30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:09.935271 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455c0d6-5e24-4a4b-9a9a-601d8dc6af30 admin admin] PATH_INFO: `/v3/projects/1bb28576ea31403da246cbfffe90e7db` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:09.936529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455c0d6-5e24-4a4b-9a9a-601d8dc6af30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bb28576ea31403da246cbfffe90e7db)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:09.938704 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455c0d6-5e24-4a4b-9a9a-601d8dc6af30 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:09.998778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b455c0d6-5e24-4a4b-9a9a-601d8dc6af30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bb28576ea31403da246cbfffe90e7db', 'request_id': 'req-b455c0d6-5e24-4a4b-9a9a-601d8dc6af30'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:10.005792 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b455c0d6-5e24-4a4b-9a9a-601d8dc6af30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bb28576ea31403da246cbfffe90e7db', 'request_id': 'req-b455c0d6-5e24-4a4b-9a9a-601d8dc6af30'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:10.014837 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b455c0d6-5e24-4a4b-9a9a-601d8dc6af30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bb28576ea31403da246cbfffe90e7db', 'request_id': 'req-b455c0d6-5e24-4a4b-9a9a-601d8dc6af30'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:10.030123 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b455c0d6-5e24-4a4b-9a9a-601d8dc6af30 admin admin] The token cache is being invalidate because project 1bb28576ea31403da246cbfffe90e7db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:48:10.030427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b455c0d6-5e24-4a4b-9a9a-601d8dc6af30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:10.031102 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b455c0d6-5e24-4a4b-9a9a-601d8dc6af30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:10.032799 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1259/2496] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:48:09 2022] DELETE /identity/v3/projects/1bb28576ea31403da246cbfffe90e7db => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:10.037807 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0167a5-8deb-4578-b783-9245e90e1858 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:10.056390 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0167a5-8deb-4578-b783-9245e90e1858 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:10.099950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455c0d6-5e24-4a4b-9a9a-601d8dc6af30 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:10.101396 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b455c0d6-5e24-4a4b-9a9a-601d8dc6af30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:10.129221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ccf7cc-e0ed-4302-9680-ae8e64c8782e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:10.129883 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccf7cc-e0ed-4302-9680-ae8e64c8782e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:10.130181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccf7cc-e0ed-4302-9680-ae8e64c8782e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:10.130457 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccf7cc-e0ed-4302-9680-ae8e64c8782e admin admin] PATH_INFO: `/v3/users/dc9390b9dbc549fd86444f1a9fead9cf` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:10.131973 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ccf7cc-e0ed-4302-9680-ae8e64c8782e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc9390b9dbc549fd86444f1a9fead9cf)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:10.133924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ccf7cc-e0ed-4302-9680-ae8e64c8782e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:10.160084 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:10.168819 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455c0d6-5e24-4a4b-9a9a-601d8dc6af30 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:10.173307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea145c-ce2f-40c4-b853-910bf93602da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:10.173945 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea145c-ce2f-40c4-b853-910bf93602da service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:10.174211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea145c-ce2f-40c4-b853-910bf93602da service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:10.174526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea145c-ce2f-40c4-b853-910bf93602da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:10.176398 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-24ea145c-ce2f-40c4-b853-910bf93602da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:10.188725 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a2ccf7cc-e0ed-4302-9680-ae8e64c8782e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc9390b9dbc549fd86444f1a9fead9cf'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:10.197451 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a2ccf7cc-e0ed-4302-9680-ae8e64c8782e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc9390b9dbc549fd86444f1a9fead9cf'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:10.198353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a2ccf7cc-e0ed-4302-9680-ae8e64c8782e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc9390b9dbc549fd86444f1a9fead9cf'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:10.207886 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a2ccf7cc-e0ed-4302-9680-ae8e64c8782e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc9390b9dbc549fd86444f1a9fead9cf'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:10.208689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a2ccf7cc-e0ed-4302-9680-ae8e64c8782e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc9390b9dbc549fd86444f1a9fead9cf'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:10.216891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:10.233038 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1238/2497] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:48:10 2022] DELETE /identity/v3/users/dc9390b9dbc549fd86444f1a9fead9cf => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:10.250680 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea145c-ce2f-40c4-b853-910bf93602da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:10.252094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea145c-ce2f-40c4-b853-910bf93602da service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:10.262221 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1260/2498] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:48:10.549490 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ccf7cc-e0ed-4302-9680-ae8e64c8782e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:10.550946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a2ccf7cc-e0ed-4302-9680-ae8e64c8782e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:10.626646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ccf7cc-e0ed-4302-9680-ae8e64c8782e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:10.632367 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09bb6e-ce2e-48f0-90a5-115aa997747e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:10.633023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09bb6e-ce2e-48f0-90a5-115aa997747e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:10.633409 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09bb6e-ce2e-48f0-90a5-115aa997747e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:10.633683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09bb6e-ce2e-48f0-90a5-115aa997747e admin admin] PATH_INFO: `/v3/projects/d8f18c796d814a948be8754c4dc52f8f` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:10.634917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c09bb6e-ce2e-48f0-90a5-115aa997747e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8f18c796d814a948be8754c4dc52f8f)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:10.637033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c09bb6e-ce2e-48f0-90a5-115aa997747e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:10.690903 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4c09bb6e-ce2e-48f0-90a5-115aa997747e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8f18c796d814a948be8754c4dc52f8f', 'request_id': 'req-4c09bb6e-ce2e-48f0-90a5-115aa997747e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:10.697303 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4c09bb6e-ce2e-48f0-90a5-115aa997747e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8f18c796d814a948be8754c4dc52f8f', 'request_id': 'req-4c09bb6e-ce2e-48f0-90a5-115aa997747e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:10.706005 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4c09bb6e-ce2e-48f0-90a5-115aa997747e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8f18c796d814a948be8754c4dc52f8f', 'request_id': 'req-4c09bb6e-ce2e-48f0-90a5-115aa997747e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:10.718986 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4c09bb6e-ce2e-48f0-90a5-115aa997747e admin admin] The token cache is being invalidate because project d8f18c796d814a948be8754c4dc52f8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:48:10.719289 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4c09bb6e-ce2e-48f0-90a5-115aa997747e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:10.719974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4c09bb6e-ce2e-48f0-90a5-115aa997747e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:10.721839 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1239/2499] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:48:10 2022] DELETE /identity/v3/projects/d8f18c796d814a948be8754c4dc52f8f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:10.727211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf75b54-eb3f-4ad2-a7e9-a808a3045c66 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:10.727501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf75b54-eb3f-4ad2-a7e9-a808a3045c66 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:10.727804 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf75b54-eb3f-4ad2-a7e9-a808a3045c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:10.748561 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-2bf75b54-eb3f-4ad2-a7e9-a808a3045c66 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:10.749852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-2bf75b54-eb3f-4ad2-a7e9-a808a3045c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:10.811990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1261/2500] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:48:10.820943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41833eaf-2cfb-460e-a533-5da3d28a90ed None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:10.821258 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41833eaf-2cfb-460e-a533-5da3d28a90ed None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:10.821535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41833eaf-2cfb-460e-a533-5da3d28a90ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:10.843090 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-41833eaf-2cfb-460e-a533-5da3d28a90ed None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:10.844330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-41833eaf-2cfb-460e-a533-5da3d28a90ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:10.853159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1240/2501] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:48:10.858131 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf75b54-eb3f-4ad2-a7e9-a808a3045c66 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:10.867791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf75b54-eb3f-4ad2-a7e9-a808a3045c66 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:10.872913 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0aff6c-2469-4232-a397-a20b15b13bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:10.873694 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0aff6c-2469-4232-a397-a20b15b13bcb admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:10.873982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0aff6c-2469-4232-a397-a20b15b13bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:10.874259 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0aff6c-2469-4232-a397-a20b15b13bcb admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:10.874995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0aff6c-2469-4232-a397-a20b15b13bcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:48:10.875330 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0aff6c-2469-4232-a397-a20b15b13bcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:10.876995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0aff6c-2469-4232-a397-a20b15b13bcb admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:10.890190 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1262/2502] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:48:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:10.894745 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41833eaf-2cfb-460e-a533-5da3d28a90ed None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:10.905016 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41833eaf-2cfb-460e-a533-5da3d28a90ed None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:10.910063 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10d43f-e96f-4b8e-af73-f654ff525f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:10.910699 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10d43f-e96f-4b8e-af73-f654ff525f38 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:10.910981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10d43f-e96f-4b8e-af73-f654ff525f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:10.911258 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10d43f-e96f-4b8e-af73-f654ff525f38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:10.911943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10d43f-e96f-4b8e-af73-f654ff525f38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:10.913950 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10d43f-e96f-4b8e-af73-f654ff525f38 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:10.942937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1241/2503] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:48:10 2022] POST /identity/v3/projects => generated 380 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:10.947069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0aff6c-2469-4232-a397-a20b15b13bcb admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:10.958166 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0aff6c-2469-4232-a397-a20b15b13bcb admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:11.015579 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39f51e-7553-4ab7-b87b-553ee9184207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:11.016341 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39f51e-7553-4ab7-b87b-553ee9184207 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:11.016532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39f51e-7553-4ab7-b87b-553ee9184207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:11.016723 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39f51e-7553-4ab7-b87b-553ee9184207 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:11.017640 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39f51e-7553-4ab7-b87b-553ee9184207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:11.019335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39f51e-7553-4ab7-b87b-553ee9184207 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:11.042257 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1263/2504] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:48:10 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:11.046400 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10d43f-e96f-4b8e-af73-f654ff525f38 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:11.058200 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10d43f-e96f-4b8e-af73-f654ff525f38 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:11.063228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c7788-b3c1-48e8-af52-733a3e5b89c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:11.063973 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c7788-b3c1-48e8-af52-733a3e5b89c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:11.064300 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c7788-b3c1-48e8-af52-733a3e5b89c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:11.064625 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c7788-b3c1-48e8-af52-733a3e5b89c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:11.065349 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c7788-b3c1-48e8-af52-733a3e5b89c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:11.066939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c7788-b3c1-48e8-af52-733a3e5b89c5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:11.078319 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1242/2505] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:11 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:11.083144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39f51e-7553-4ab7-b87b-553ee9184207 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:11.094735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39f51e-7553-4ab7-b87b-553ee9184207 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:11.099688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b879ea-aa76-45bd-ba5b-ed6a28b54be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:11.100299 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b879ea-aa76-45bd-ba5b-ed6a28b54be5 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:11.100580 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b879ea-aa76-45bd-ba5b-ed6a28b54be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:11.100965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b879ea-aa76-45bd-ba5b-ed6a28b54be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:11.101761 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b879ea-aa76-45bd-ba5b-ed6a28b54be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:11.103351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b879ea-aa76-45bd-ba5b-ed6a28b54be5 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:11.114603 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1264/2506] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:11 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:11.118985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c7788-b3c1-48e8-af52-733a3e5b89c5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:11.129791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c7788-b3c1-48e8-af52-733a3e5b89c5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:11.135885 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb0dfe-ea58-4bbf-9554-2afc4e77f17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:11.136771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb0dfe-ea58-4bbf-9554-2afc4e77f17c admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:11.136971 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb0dfe-ea58-4bbf-9554-2afc4e77f17c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:11.137310 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb0dfe-ea58-4bbf-9554-2afc4e77f17c admin admin] PATH_INFO: `/v3/projects/20a573d9fff64e039910daeab7f953dc/users/fa5a75f6d32546dda745bbfc31baa024/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:11.154510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb0dfe-ea58-4bbf-9554-2afc4e77f17c admin admin] RBAC: Authorizing `identity:create_grant(project_id=20a573d9fff64e039910daeab7f953dc, user_id=fa5a75f6d32546dda745bbfc31baa024, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:11.157231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb0dfe-ea58-4bbf-9554-2afc4e77f17c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:11.171235 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1243/2507] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:11 2022] PUT /identity/v3/projects/20a573d9fff64e039910daeab7f953dc/users/fa5a75f6d32546dda745bbfc31baa024/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:11.182016 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ffd2e-9717-4d1f-81d6-0fc56943838d None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:11.182305 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ffd2e-9717-4d1f-81d6-0fc56943838d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:11.182584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ffd2e-9717-4d1f-81d6-0fc56943838d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:11.190074 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:11.213064 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-8d1ffd2e-9717-4d1f-81d6-0fc56943838d None None] MFA Rules not processed for user `fa5a75f6d32546dda745bbfc31baa024`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:11.214387 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-8d1ffd2e-9717-4d1f-81d6-0fc56943838d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:11.285885 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1265/2508] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:11 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:11.293369 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb0dfe-ea58-4bbf-9554-2afc4e77f17c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:11.313028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:11.363632 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb0dfe-ea58-4bbf-9554-2afc4e77f17c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:11.368570 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e637d-b973-4682-bed6-f205242c9d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:11.369214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e637d-b973-4682-bed6-f205242c9d62 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:11.369500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e637d-b973-4682-bed6-f205242c9d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:11.370320 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e637d-b973-4682-bed6-f205242c9d62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:11.371592 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e637d-b973-4682-bed6-f205242c9d62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:11.373539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e637d-b973-4682-bed6-f205242c9d62 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:11.405247 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1244/2509] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:48:11 2022] POST /identity/v3/projects => generated 382 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:11.406863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ffd2e-9717-4d1f-81d6-0fc56943838d None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:11.418029 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ffd2e-9717-4d1f-81d6-0fc56943838d None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:11.488356 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ef2ca-34d4-4df3-853a-71daf123973e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:11.488989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ef2ca-34d4-4df3-853a-71daf123973e admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:11.489302 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ef2ca-34d4-4df3-853a-71daf123973e admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:11.489578 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ef2ca-34d4-4df3-853a-71daf123973e admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:11.490261 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ef2ca-34d4-4df3-853a-71daf123973e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:11.492255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ef2ca-34d4-4df3-853a-71daf123973e admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:11.517916 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1266/2510] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:48:11 2022] POST /identity/v3/users => generated 298 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:11.521764 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e637d-b973-4682-bed6-f205242c9d62 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:11.533420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e637d-b973-4682-bed6-f205242c9d62 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:11.538476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b1509-a754-4977-ac3a-7b656e23de8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:11.539218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b1509-a754-4977-ac3a-7b656e23de8f admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:11.539615 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b1509-a754-4977-ac3a-7b656e23de8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:11.540024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b1509-a754-4977-ac3a-7b656e23de8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:11.540833 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b1509-a754-4977-ac3a-7b656e23de8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:11.543380 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b1509-a754-4977-ac3a-7b656e23de8f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:11.557641 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1245/2511] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:11 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:48:11.560251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ef2ca-34d4-4df3-853a-71daf123973e admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:11.570634 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ef2ca-34d4-4df3-853a-71daf123973e admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:11.575985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3e92d-f941-490a-9d8e-c22c3796702d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:11.576687 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3e92d-f941-490a-9d8e-c22c3796702d admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:11.577007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3e92d-f941-490a-9d8e-c22c3796702d admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:11.577317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3e92d-f941-490a-9d8e-c22c3796702d admin admin] PATH_INFO: `/v3/projects/a9a059627978403b8587ea8f429e8fa0/users/71bcd1f6d4874faaade023890d45430b/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:11.593041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3e92d-f941-490a-9d8e-c22c3796702d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a059627978403b8587ea8f429e8fa0, user_id=71bcd1f6d4874faaade023890d45430b, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:11.595743 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3e92d-f941-490a-9d8e-c22c3796702d admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:11.610492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1267/2512] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:11 2022] PUT /identity/v3/projects/a9a059627978403b8587ea8f429e8fa0/users/71bcd1f6d4874faaade023890d45430b/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:11.614739 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b1509-a754-4977-ac3a-7b656e23de8f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:11.626379 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b1509-a754-4977-ac3a-7b656e23de8f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:11.691727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426af80-8f73-4911-a696-54dfe8a76d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:11.692361 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426af80-8f73-4911-a696-54dfe8a76d62 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:11.692649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426af80-8f73-4911-a696-54dfe8a76d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:11.692932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426af80-8f73-4911-a696-54dfe8a76d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:11.693661 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426af80-8f73-4911-a696-54dfe8a76d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:11.695454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426af80-8f73-4911-a696-54dfe8a76d62 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:11.708929 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1246/2513] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:11 2022] GET /identity/v3/roles => generated 1427 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:48:11.713165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3e92d-f941-490a-9d8e-c22c3796702d admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:11.724872 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3e92d-f941-490a-9d8e-c22c3796702d admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:11.731191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9a00f-8aa9-4bf1-b13a-b6e2613cb1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:11.731929 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9a00f-8aa9-4bf1-b13a-b6e2613cb1c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:11.732508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9a00f-8aa9-4bf1-b13a-b6e2613cb1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:11.732893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9a00f-8aa9-4bf1-b13a-b6e2613cb1c3 admin admin] PATH_INFO: `/v3/projects/a9a059627978403b8587ea8f429e8fa0/users/71bcd1f6d4874faaade023890d45430b/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:11.735157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9a00f-8aa9-4bf1-b13a-b6e2613cb1c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a059627978403b8587ea8f429e8fa0, user_id=71bcd1f6d4874faaade023890d45430b, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:11.737841 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9a00f-8aa9-4bf1-b13a-b6e2613cb1c3 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:11.749777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1268/2514] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:11 2022] PUT /identity/v3/projects/a9a059627978403b8587ea8f429e8fa0/users/71bcd1f6d4874faaade023890d45430b/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:11.761021 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34835b07-beeb-4051-ad29-deb4a3b2061e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:11.761363 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34835b07-beeb-4051-ad29-deb4a3b2061e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:11.761618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34835b07-beeb-4051-ad29-deb4a3b2061e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:11.798810 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-34835b07-beeb-4051-ad29-deb4a3b2061e None None] MFA Rules not processed for user `71bcd1f6d4874faaade023890d45430b`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:11.799974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-34835b07-beeb-4051-ad29-deb4a3b2061e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:11.881453 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1247/2515] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:11 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:11.898373 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9a00f-8aa9-4bf1-b13a-b6e2613cb1c3 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:11.900088 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-aae9a00f-8aa9-4bf1-b13a-b6e2613cb1c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:11.952846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:11.967321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9a00f-8aa9-4bf1-b13a-b6e2613cb1c3 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:11.970965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849ffdb-e5b0-4e8a-8c4b-4ede121f51c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:11.971573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849ffdb-e5b0-4e8a-8c4b-4ede121f51c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:11.971851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849ffdb-e5b0-4e8a-8c4b-4ede121f51c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:11.972122 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849ffdb-e5b0-4e8a-8c4b-4ede121f51c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:11.981215 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849ffdb-e5b0-4e8a-8c4b-4ede121f51c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:11.982634 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849ffdb-e5b0-4e8a-8c4b-4ede121f51c5 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:11.992612 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1269/2516] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:11 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:48:12.317606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34835b07-beeb-4051-ad29-deb4a3b2061e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:12.328200 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34835b07-beeb-4051-ad29-deb4a3b2061e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:12.332879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951b219-888d-445a-991c-300d14c27264 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71bcd1f6d4874faaade023890d45430b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a059627978403b8587ea8f429e8fa0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:12.333533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951b219-888d-445a-991c-300d14c27264 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:12.333823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951b219-888d-445a-991c-300d14c27264 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:12.334104 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951b219-888d-445a-991c-300d14c27264 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:12.334796 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951b219-888d-445a-991c-300d14c27264 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:48:12.335106 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951b219-888d-445a-991c-300d14c27264 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:12.336691 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951b219-888d-445a-991c-300d14c27264 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:12.343605 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:12.350560 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1248/2517] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:48:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:12.354932 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849ffdb-e5b0-4e8a-8c4b-4ede121f51c5 service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:12.363996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849ffdb-e5b0-4e8a-8c4b-4ede121f51c5 service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:12.368534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2614f25-03df-45ed-aaaa-dfb41a281b5e tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71bcd1f6d4874faaade023890d45430b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a059627978403b8587ea8f429e8fa0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:12.369163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2614f25-03df-45ed-aaaa-dfb41a281b5e tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:12.369453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2614f25-03df-45ed-aaaa-dfb41a281b5e tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:12.370848 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2614f25-03df-45ed-aaaa-dfb41a281b5e tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:12.371524 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2614f25-03df-45ed-aaaa-dfb41a281b5e tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:12.373442 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2614f25-03df-45ed-aaaa-dfb41a281b5e tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:12.401776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1270/2518] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:48:12 2022] POST /identity/v3/projects => generated 355 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:12.405926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951b219-888d-445a-991c-300d14c27264 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:12.415807 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951b219-888d-445a-991c-300d14c27264 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:12.475961 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d2c6a-5ed6-4943-95fd-6cbd8938d9d1 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71bcd1f6d4874faaade023890d45430b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a059627978403b8587ea8f429e8fa0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:12.476501 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d2c6a-5ed6-4943-95fd-6cbd8938d9d1 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:12.476790 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d2c6a-5ed6-4943-95fd-6cbd8938d9d1 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:12.477070 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d2c6a-5ed6-4943-95fd-6cbd8938d9d1 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:12.477802 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d2c6a-5ed6-4943-95fd-6cbd8938d9d1 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:48:12.478142 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d2c6a-5ed6-4943-95fd-6cbd8938d9d1 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:12.480339 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d2c6a-5ed6-4943-95fd-6cbd8938d9d1 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:12.492250 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1249/2519] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:48:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:12.534538 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2614f25-03df-45ed-aaaa-dfb41a281b5e tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:12.545284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2614f25-03df-45ed-aaaa-dfb41a281b5e tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:12.550199 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36737c-d853-42db-9092-f0a398c844b8 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71bcd1f6d4874faaade023890d45430b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a059627978403b8587ea8f429e8fa0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:12.550830 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36737c-d853-42db-9092-f0a398c844b8 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:12.551109 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36737c-d853-42db-9092-f0a398c844b8 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:12.551382 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36737c-d853-42db-9092-f0a398c844b8 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] PATH_INFO: `/v3/projects/dfa7aa3b6b8a41729a3d1224947d94d4` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:12.552524 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36737c-d853-42db-9092-f0a398c844b8 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] RBAC: Authorizing `identity:delete_project(project_id=dfa7aa3b6b8a41729a3d1224947d94d4)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:12.554443 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36737c-d853-42db-9092-f0a398c844b8 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:12.603488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5e36737c-d853-42db-9092-f0a398c844b8 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfa7aa3b6b8a41729a3d1224947d94d4', 'request_id': 'req-5e36737c-d853-42db-9092-f0a398c844b8'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:12.611005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5e36737c-d853-42db-9092-f0a398c844b8 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfa7aa3b6b8a41729a3d1224947d94d4', 'request_id': 'req-5e36737c-d853-42db-9092-f0a398c844b8'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:12.618545 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5e36737c-d853-42db-9092-f0a398c844b8 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfa7aa3b6b8a41729a3d1224947d94d4', 'request_id': 'req-5e36737c-d853-42db-9092-f0a398c844b8'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:12.629945 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5e36737c-d853-42db-9092-f0a398c844b8 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] The token cache is being invalidate because project dfa7aa3b6b8a41729a3d1224947d94d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:48:12.630236 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5e36737c-d853-42db-9092-f0a398c844b8 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:12.631166 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5e36737c-d853-42db-9092-f0a398c844b8 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:12.632684 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1271/2520] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:48:12 2022] DELETE /identity/v3/projects/dfa7aa3b6b8a41729a3d1224947d94d4 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:12.830924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d2c6a-5ed6-4943-95fd-6cbd8938d9d1 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:12.834117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ae3d2c6a-5ed6-4943-95fd-6cbd8938d9d1 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:12.925538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d2c6a-5ed6-4943-95fd-6cbd8938d9d1 tempest-VolumeQuotaClassesTest-1243487812 tempest-VolumeQuotaClassesTest-1243487812-project-admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:12.992853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a2634-f5d8-4e77-a16b-9d6fa79f0cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:12.993548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a2634-f5d8-4e77-a16b-9d6fa79f0cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:12.993831 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a2634-f5d8-4e77-a16b-9d6fa79f0cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:12.994098 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a2634-f5d8-4e77-a16b-9d6fa79f0cd0 admin admin] PATH_INFO: `/v3/users/fa5a75f6d32546dda745bbfc31baa024` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:12.995799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a2634-f5d8-4e77-a16b-9d6fa79f0cd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa5a75f6d32546dda745bbfc31baa024)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:12.997762 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a2634-f5d8-4e77-a16b-9d6fa79f0cd0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:13.050846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:13.058068 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-626a2634-f5d8-4e77-a16b-9d6fa79f0cd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa5a75f6d32546dda745bbfc31baa024'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:13.069394 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-626a2634-f5d8-4e77-a16b-9d6fa79f0cd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa5a75f6d32546dda745bbfc31baa024'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:13.070627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-626a2634-f5d8-4e77-a16b-9d6fa79f0cd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa5a75f6d32546dda745bbfc31baa024'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:13.078472 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-626a2634-f5d8-4e77-a16b-9d6fa79f0cd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa5a75f6d32546dda745bbfc31baa024'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:13.079496 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-626a2634-f5d8-4e77-a16b-9d6fa79f0cd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa5a75f6d32546dda745bbfc31baa024'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:13.103617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1250/2521] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:48:12 2022] DELETE /identity/v3/users/fa5a75f6d32546dda745bbfc31baa024 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:13.109069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f39eaa-ec9a-4473-852d-b218f8c74087 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:13.109541 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f39eaa-ec9a-4473-852d-b218f8c74087 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:13.109731 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f39eaa-ec9a-4473-852d-b218f8c74087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:13.129527 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-85f39eaa-ec9a-4473-852d-b218f8c74087 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:13.131051 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-85f39eaa-ec9a-4473-852d-b218f8c74087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:13.195626 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1272/2522] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:13.208767 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a2634-f5d8-4e77-a16b-9d6fa79f0cd0 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:13.210639 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-626a2634-f5d8-4e77-a16b-9d6fa79f0cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:13.295736 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a2634-f5d8-4e77-a16b-9d6fa79f0cd0 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:13.300737 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae05c7-0cee-4846-8591-c573cbd03fff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:13.301463 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae05c7-0cee-4846-8591-c573cbd03fff service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:13.301764 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae05c7-0cee-4846-8591-c573cbd03fff service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:13.302055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae05c7-0cee-4846-8591-c573cbd03fff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:13.312875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae05c7-0cee-4846-8591-c573cbd03fff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:13.314804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae05c7-0cee-4846-8591-c573cbd03fff service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:13.338342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1251/2523] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:13 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:48:13.690031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f39eaa-ec9a-4473-852d-b218f8c74087 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:13.691165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-85f39eaa-ec9a-4473-852d-b218f8c74087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:13.703831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f39eaa-ec9a-4473-852d-b218f8c74087 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:13.708447 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9a7e2-694a-4d09-9295-bebb66e2706c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:13.709089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9a7e2-694a-4d09-9295-bebb66e2706c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:13.709402 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9a7e2-694a-4d09-9295-bebb66e2706c admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:13.709671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9a7e2-694a-4d09-9295-bebb66e2706c admin admin] PATH_INFO: `/v3/projects/20a573d9fff64e039910daeab7f953dc` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:13.710823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9a7e2-694a-4d09-9295-bebb66e2706c admin admin] RBAC: Authorizing `identity:delete_project(project_id=20a573d9fff64e039910daeab7f953dc)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:13.712741 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9a7e2-694a-4d09-9295-bebb66e2706c admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:13.759398 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:13.764095 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-76f9a7e2-694a-4d09-9295-bebb66e2706c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20a573d9fff64e039910daeab7f953dc', 'request_id': 'req-76f9a7e2-694a-4d09-9295-bebb66e2706c'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:13.771162 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-76f9a7e2-694a-4d09-9295-bebb66e2706c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20a573d9fff64e039910daeab7f953dc', 'request_id': 'req-76f9a7e2-694a-4d09-9295-bebb66e2706c'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:13.778996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-76f9a7e2-694a-4d09-9295-bebb66e2706c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20a573d9fff64e039910daeab7f953dc', 'request_id': 'req-76f9a7e2-694a-4d09-9295-bebb66e2706c'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:13.790788 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-76f9a7e2-694a-4d09-9295-bebb66e2706c admin admin] The token cache is being invalidate because project 20a573d9fff64e039910daeab7f953dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:48:13.791105 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-76f9a7e2-694a-4d09-9295-bebb66e2706c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:13.791797 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-76f9a7e2-694a-4d09-9295-bebb66e2706c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:13.793351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1273/2524] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:48:13 2022] DELETE /identity/v3/projects/20a573d9fff64e039910daeab7f953dc => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:13.797442 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae05c7-0cee-4846-8591-c573cbd03fff service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:13.808075 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae05c7-0cee-4846-8591-c573cbd03fff service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:13.877061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc98dc6-5146-40e2-9c8e-794d29dc2dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:13.877555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc98dc6-5146-40e2-9c8e-794d29dc2dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:13.877825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc98dc6-5146-40e2-9c8e-794d29dc2dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:13.878104 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc98dc6-5146-40e2-9c8e-794d29dc2dfd admin admin] PATH_INFO: `/v3/users/71bcd1f6d4874faaade023890d45430b` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:13.879310 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc98dc6-5146-40e2-9c8e-794d29dc2dfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=71bcd1f6d4874faaade023890d45430b)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:13.881556 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc98dc6-5146-40e2-9c8e-794d29dc2dfd admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:13.941617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9cc98dc6-5146-40e2-9c8e-794d29dc2dfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71bcd1f6d4874faaade023890d45430b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:13.948597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9cc98dc6-5146-40e2-9c8e-794d29dc2dfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71bcd1f6d4874faaade023890d45430b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:13.949321 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9cc98dc6-5146-40e2-9c8e-794d29dc2dfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71bcd1f6d4874faaade023890d45430b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:13.957098 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9cc98dc6-5146-40e2-9c8e-794d29dc2dfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71bcd1f6d4874faaade023890d45430b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:13.957790 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9cc98dc6-5146-40e2-9c8e-794d29dc2dfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71bcd1f6d4874faaade023890d45430b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:13.959064 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:13.979741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1252/2525] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:48:13 2022] DELETE /identity/v3/users/71bcd1f6d4874faaade023890d45430b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:14.284964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9a7e2-694a-4d09-9295-bebb66e2706c admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:14.290604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-76f9a7e2-694a-4d09-9295-bebb66e2706c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:14.309063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9a7e2-694a-4d09-9295-bebb66e2706c admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:14.388196 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d57bce-053d-4247-84ab-749a8aab81fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:14.388945 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d57bce-053d-4247-84ab-749a8aab81fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:14.389333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d57bce-053d-4247-84ab-749a8aab81fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:14.389811 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d57bce-053d-4247-84ab-749a8aab81fb admin admin] PATH_INFO: `/v3/projects/a9a059627978403b8587ea8f429e8fa0` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:14.391308 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d57bce-053d-4247-84ab-749a8aab81fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9a059627978403b8587ea8f429e8fa0)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:14.393340 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d57bce-053d-4247-84ab-749a8aab81fb admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:14.469090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-25d57bce-053d-4247-84ab-749a8aab81fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9a059627978403b8587ea8f429e8fa0', 'request_id': 'req-25d57bce-053d-4247-84ab-749a8aab81fb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:14.477782 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-25d57bce-053d-4247-84ab-749a8aab81fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9a059627978403b8587ea8f429e8fa0', 'request_id': 'req-25d57bce-053d-4247-84ab-749a8aab81fb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:14.488265 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-25d57bce-053d-4247-84ab-749a8aab81fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9a059627978403b8587ea8f429e8fa0', 'request_id': 'req-25d57bce-053d-4247-84ab-749a8aab81fb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:14.500305 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-25d57bce-053d-4247-84ab-749a8aab81fb admin admin] The token cache is being invalidate because project a9a059627978403b8587ea8f429e8fa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:48:14.500631 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-25d57bce-053d-4247-84ab-749a8aab81fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:14.501937 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-25d57bce-053d-4247-84ab-749a8aab81fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:14.503725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1274/2526] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:48:14 2022] DELETE /identity/v3/projects/a9a059627978403b8587ea8f429e8fa0 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:14.508890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230857e-0b2c-4f3d-a3c8-49919f56b680 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:14.510552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230857e-0b2c-4f3d-a3c8-49919f56b680 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:14.511075 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230857e-0b2c-4f3d-a3c8-49919f56b680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:14.545834 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d230857e-0b2c-4f3d-a3c8-49919f56b680 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:14.547072 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d230857e-0b2c-4f3d-a3c8-49919f56b680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:14.643954 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1253/2527] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:48:14.653158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28eca57-346c-4b10-a8cc-64a3bab588e8 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:14.653434 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28eca57-346c-4b10-a8cc-64a3bab588e8 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:14.653739 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28eca57-346c-4b10-a8cc-64a3bab588e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:14.674379 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-d28eca57-346c-4b10-a8cc-64a3bab588e8 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:14.675600 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d28eca57-346c-4b10-a8cc-64a3bab588e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:14.684372 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1275/2528] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:14.688738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230857e-0b2c-4f3d-a3c8-49919f56b680 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:14.699901 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230857e-0b2c-4f3d-a3c8-49919f56b680 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:14.704720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48a3d1-4fc4-4391-a14a-30d22246409d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:14.705384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48a3d1-4fc4-4391-a14a-30d22246409d admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:14.705669 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48a3d1-4fc4-4391-a14a-30d22246409d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:14.705950 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48a3d1-4fc4-4391-a14a-30d22246409d admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:14.706645 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48a3d1-4fc4-4391-a14a-30d22246409d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:48:14.706953 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48a3d1-4fc4-4391-a14a-30d22246409d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:14.708546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48a3d1-4fc4-4391-a14a-30d22246409d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:14.721417 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1254/2529] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:48:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:14.725860 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28eca57-346c-4b10-a8cc-64a3bab588e8 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:14.736392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28eca57-346c-4b10-a8cc-64a3bab588e8 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:14.741409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c2a01-232f-45c6-97bc-7462f34cbaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:14.742018 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c2a01-232f-45c6-97bc-7462f34cbaad admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:14.742301 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c2a01-232f-45c6-97bc-7462f34cbaad admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:14.742579 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c2a01-232f-45c6-97bc-7462f34cbaad admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:14.743238 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c2a01-232f-45c6-97bc-7462f34cbaad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:14.745898 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c2a01-232f-45c6-97bc-7462f34cbaad admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:14.773679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1276/2530] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:48:14 2022] POST /identity/v3/projects => generated 386 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:14.778602 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48a3d1-4fc4-4391-a14a-30d22246409d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:14.788926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48a3d1-4fc4-4391-a14a-30d22246409d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:14.851013 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472bbe0-6b51-413f-b2ba-4ad38f26ffef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:14.851558 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472bbe0-6b51-413f-b2ba-4ad38f26ffef admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:14.851846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472bbe0-6b51-413f-b2ba-4ad38f26ffef admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:14.852125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472bbe0-6b51-413f-b2ba-4ad38f26ffef admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:14.852808 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472bbe0-6b51-413f-b2ba-4ad38f26ffef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:14.855367 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472bbe0-6b51-413f-b2ba-4ad38f26ffef admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:14.878458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1255/2531] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:48:14 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:14.882904 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c2a01-232f-45c6-97bc-7462f34cbaad admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:14.893927 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c2a01-232f-45c6-97bc-7462f34cbaad admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:14.901453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545d081-ad63-45e0-86d8-e2a5711d12a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:14.902079 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545d081-ad63-45e0-86d8-e2a5711d12a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:14.902359 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545d081-ad63-45e0-86d8-e2a5711d12a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:14.902633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545d081-ad63-45e0-86d8-e2a5711d12a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:14.903332 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545d081-ad63-45e0-86d8-e2a5711d12a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:14.904946 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545d081-ad63-45e0-86d8-e2a5711d12a2 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:14.915914 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1277/2532] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:14 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:14.920961 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472bbe0-6b51-413f-b2ba-4ad38f26ffef admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:14.932847 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472bbe0-6b51-413f-b2ba-4ad38f26ffef admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:14.938209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0633d-4a3b-4dd1-a1d2-d8dc47564ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:14.938848 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0633d-4a3b-4dd1-a1d2-d8dc47564ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:14.939132 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0633d-4a3b-4dd1-a1d2-d8dc47564ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:14.939412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0633d-4a3b-4dd1-a1d2-d8dc47564ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:14.940120 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0633d-4a3b-4dd1-a1d2-d8dc47564ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:14.942147 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0633d-4a3b-4dd1-a1d2-d8dc47564ea4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:14.955641 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1256/2533] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:14 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:14.963561 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545d081-ad63-45e0-86d8-e2a5711d12a2 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:14.974204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545d081-ad63-45e0-86d8-e2a5711d12a2 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:14.979443 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706550ab-fa03-4a36-be91-45461626a4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:14.980130 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706550ab-fa03-4a36-be91-45461626a4b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:14.980415 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706550ab-fa03-4a36-be91-45461626a4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:14.980693 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706550ab-fa03-4a36-be91-45461626a4b1 admin admin] PATH_INFO: `/v3/projects/a8f6d1f35f45461d8f0fada0377a40ec/users/7c91ac47ab3a4a0e90e9d4c3a064237e/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:14.996041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706550ab-fa03-4a36-be91-45461626a4b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8f6d1f35f45461d8f0fada0377a40ec, user_id=7c91ac47ab3a4a0e90e9d4c3a064237e, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:14.998779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706550ab-fa03-4a36-be91-45461626a4b1 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:15.006966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:15.013398 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1278/2534] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:14 2022] PUT /identity/v3/projects/a8f6d1f35f45461d8f0fada0377a40ec/users/7c91ac47ab3a4a0e90e9d4c3a064237e/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:15.024051 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b91f4-b373-4387-a6a2-d7892ebefcc9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:15.024350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b91f4-b373-4387-a6a2-d7892ebefcc9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:15.024633 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b91f4-b373-4387-a6a2-d7892ebefcc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:15.055182 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-7e7b91f4-b373-4387-a6a2-d7892ebefcc9 None None] MFA Rules not processed for user `7c91ac47ab3a4a0e90e9d4c3a064237e`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:15.056439 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7e7b91f4-b373-4387-a6a2-d7892ebefcc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:15.091144 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:15.128051 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1257/2535] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:15 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:15.132750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706550ab-fa03-4a36-be91-45461626a4b1 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:15.196592 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706550ab-fa03-4a36-be91-45461626a4b1 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:15.201429 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d90be1-b555-4ddb-9f4d-7c38ea847a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:15.202057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d90be1-b555-4ddb-9f4d-7c38ea847a5b admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:15.202343 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d90be1-b555-4ddb-9f4d-7c38ea847a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:15.202623 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d90be1-b555-4ddb-9f4d-7c38ea847a5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:15.203295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d90be1-b555-4ddb-9f4d-7c38ea847a5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:15.205249 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d90be1-b555-4ddb-9f4d-7c38ea847a5b admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:15.235723 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1279/2536] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:48:15 2022] POST /identity/v3/projects => generated 386 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:15.239819 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b91f4-b373-4387-a6a2-d7892ebefcc9 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:15.250863 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b91f4-b373-4387-a6a2-d7892ebefcc9 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:15.314384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b92706-b896-4e4e-b569-918ce3709175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:15.315025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b92706-b896-4e4e-b569-918ce3709175 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:15.315299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b92706-b896-4e4e-b569-918ce3709175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:15.315571 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b92706-b896-4e4e-b569-918ce3709175 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:15.316254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b92706-b896-4e4e-b569-918ce3709175 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:15.318238 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b92706-b896-4e4e-b569-918ce3709175 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:15.341900 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1258/2537] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:48:15 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:15.346026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d90be1-b555-4ddb-9f4d-7c38ea847a5b admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:15.358623 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d90be1-b555-4ddb-9f4d-7c38ea847a5b admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:15.363394 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f95d0-0e2f-4a79-a3a8-8c739aa3aa1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:15.363987 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f95d0-0e2f-4a79-a3a8-8c739aa3aa1d admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:15.364269 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f95d0-0e2f-4a79-a3a8-8c739aa3aa1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:15.364544 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f95d0-0e2f-4a79-a3a8-8c739aa3aa1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:15.365403 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f95d0-0e2f-4a79-a3a8-8c739aa3aa1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:15.367012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f95d0-0e2f-4a79-a3a8-8c739aa3aa1d admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:15.382012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1280/2538] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:15 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:48:15.386003 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b92706-b896-4e4e-b569-918ce3709175 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:15.396151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b92706-b896-4e4e-b569-918ce3709175 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:15.401674 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38afa41-b1be-47ec-9264-2ad6ed68a358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:15.402421 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38afa41-b1be-47ec-9264-2ad6ed68a358 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:15.402711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38afa41-b1be-47ec-9264-2ad6ed68a358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:15.402995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38afa41-b1be-47ec-9264-2ad6ed68a358 admin admin] PATH_INFO: `/v3/projects/203f1f1a102541bf9102af33f4d50cdb/users/a1f74cfb72554635811f0253e3cb94a6/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:15.422593 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38afa41-b1be-47ec-9264-2ad6ed68a358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=203f1f1a102541bf9102af33f4d50cdb, user_id=a1f74cfb72554635811f0253e3cb94a6, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:15.425286 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38afa41-b1be-47ec-9264-2ad6ed68a358 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:15.439100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1259/2539] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:15 2022] PUT /identity/v3/projects/203f1f1a102541bf9102af33f4d50cdb/users/a1f74cfb72554635811f0253e3cb94a6/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:15.443149 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f95d0-0e2f-4a79-a3a8-8c739aa3aa1d admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:15.455316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f95d0-0e2f-4a79-a3a8-8c739aa3aa1d admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:15.512157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f9df3-6692-428c-a874-08262c549c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:15.512768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f9df3-6692-428c-a874-08262c549c86 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:15.513080 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f9df3-6692-428c-a874-08262c549c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:15.513382 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f9df3-6692-428c-a874-08262c549c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:15.514067 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204f9df3-6692-428c-a874-08262c549c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:15.515657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204f9df3-6692-428c-a874-08262c549c86 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:15.525302 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1281/2540] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:15 2022] GET /identity/v3/roles => generated 1427 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:15.530266 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38afa41-b1be-47ec-9264-2ad6ed68a358 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:15.540541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38afa41-b1be-47ec-9264-2ad6ed68a358 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:15.546183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d450e2-cbfb-4ace-abe0-206dfa34c341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:15.546844 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d450e2-cbfb-4ace-abe0-206dfa34c341 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:15.547121 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d450e2-cbfb-4ace-abe0-206dfa34c341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:15.547393 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d450e2-cbfb-4ace-abe0-206dfa34c341 admin admin] PATH_INFO: `/v3/projects/203f1f1a102541bf9102af33f4d50cdb/users/a1f74cfb72554635811f0253e3cb94a6/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:15.549369 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d450e2-cbfb-4ace-abe0-206dfa34c341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=203f1f1a102541bf9102af33f4d50cdb, user_id=a1f74cfb72554635811f0253e3cb94a6, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:15.551962 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d450e2-cbfb-4ace-abe0-206dfa34c341 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:15.562856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1260/2541] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:15 2022] PUT /identity/v3/projects/203f1f1a102541bf9102af33f4d50cdb/users/a1f74cfb72554635811f0253e3cb94a6/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:15.574638 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90090c-dfc4-4528-bf9b-0ab0289d4488 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:15.574927 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90090c-dfc4-4528-bf9b-0ab0289d4488 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:15.575347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90090c-dfc4-4528-bf9b-0ab0289d4488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:15.605152 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-3a90090c-dfc4-4528-bf9b-0ab0289d4488 None None] MFA Rules not processed for user `a1f74cfb72554635811f0253e3cb94a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:15.606400 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3a90090c-dfc4-4528-bf9b-0ab0289d4488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:15.667598 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:15.678428 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1282/2542] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:15 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:15.689159 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d450e2-cbfb-4ace-abe0-206dfa34c341 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:15.691004 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-35d450e2-cbfb-4ace-abe0-206dfa34c341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:15.752466 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d450e2-cbfb-4ace-abe0-206dfa34c341 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:15.756430 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477569a-4ee3-47d4-ba20-ea030a334b4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:15.757054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477569a-4ee3-47d4-ba20-ea030a334b4a service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:15.757368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477569a-4ee3-47d4-ba20-ea030a334b4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:15.757646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477569a-4ee3-47d4-ba20-ea030a334b4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:15.801322 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:15.821214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477569a-4ee3-47d4-ba20-ea030a334b4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:15.822755 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477569a-4ee3-47d4-ba20-ea030a334b4a service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:15.836420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1261/2543] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:15 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:48:20.169999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90090c-dfc4-4528-bf9b-0ab0289d4488 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:20.171412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3a90090c-dfc4-4528-bf9b-0ab0289d4488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:20.183723 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90090c-dfc4-4528-bf9b-0ab0289d4488 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:20.187568 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1717e-f9e7-44cf-af74-d8e8309dae6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:20.188168 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1717e-f9e7-44cf-af74-d8e8309dae6d service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:20.188450 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1717e-f9e7-44cf-af74-d8e8309dae6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:20.188722 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1717e-f9e7-44cf-af74-d8e8309dae6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:20.198560 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1717e-f9e7-44cf-af74-d8e8309dae6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:20.199978 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1717e-f9e7-44cf-af74-d8e8309dae6d service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:20.209584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1283/2544] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:20 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:48:23.533795 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477569a-4ee3-47d4-ba20-ea030a334b4a service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:23.535362 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8477569a-4ee3-47d4-ba20-ea030a334b4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:23.601343 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477569a-4ee3-47d4-ba20-ea030a334b4a service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:23.605982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379858a3-6818-4fe6-b48d-e9e248c38123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:23.606626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379858a3-6818-4fe6-b48d-e9e248c38123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:23.606912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379858a3-6818-4fe6-b48d-e9e248c38123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:23.607190 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379858a3-6818-4fe6-b48d-e9e248c38123 admin admin] PATH_INFO: `/v3/users/4c2cd9bacdff43fdadf0759ef8f7abcc` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:23.608453 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379858a3-6818-4fe6-b48d-e9e248c38123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c2cd9bacdff43fdadf0759ef8f7abcc)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:23.610390 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379858a3-6818-4fe6-b48d-e9e248c38123 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:23.659546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-379858a3-6818-4fe6-b48d-e9e248c38123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c2cd9bacdff43fdadf0759ef8f7abcc'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:23.666837 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-379858a3-6818-4fe6-b48d-e9e248c38123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c2cd9bacdff43fdadf0759ef8f7abcc'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:23.667801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-379858a3-6818-4fe6-b48d-e9e248c38123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c2cd9bacdff43fdadf0759ef8f7abcc'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:23.675030 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-379858a3-6818-4fe6-b48d-e9e248c38123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c2cd9bacdff43fdadf0759ef8f7abcc'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:23.675647 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-379858a3-6818-4fe6-b48d-e9e248c38123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c2cd9bacdff43fdadf0759ef8f7abcc'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:23.693689 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1262/2545] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:48:23 2022] DELETE /identity/v3/users/4c2cd9bacdff43fdadf0759ef8f7abcc => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:23.865936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1717e-f9e7-44cf-af74-d8e8309dae6d service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:23.878041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1717e-f9e7-44cf-af74-d8e8309dae6d service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:23.942430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aba7c6-08dc-48c0-9592-34bb1733a46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:23.943074 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aba7c6-08dc-48c0-9592-34bb1733a46e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:23.943357 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aba7c6-08dc-48c0-9592-34bb1733a46e admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:23.943636 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aba7c6-08dc-48c0-9592-34bb1733a46e admin admin] PATH_INFO: `/v3/projects/f99eef8b6cc64ee8aae548862078c315` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:23.944787 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aba7c6-08dc-48c0-9592-34bb1733a46e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f99eef8b6cc64ee8aae548862078c315)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:23.946727 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aba7c6-08dc-48c0-9592-34bb1733a46e admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:23.997469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-40aba7c6-08dc-48c0-9592-34bb1733a46e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f99eef8b6cc64ee8aae548862078c315', 'request_id': 'req-40aba7c6-08dc-48c0-9592-34bb1733a46e'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:24.004361 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-40aba7c6-08dc-48c0-9592-34bb1733a46e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f99eef8b6cc64ee8aae548862078c315', 'request_id': 'req-40aba7c6-08dc-48c0-9592-34bb1733a46e'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:24.013988 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-40aba7c6-08dc-48c0-9592-34bb1733a46e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f99eef8b6cc64ee8aae548862078c315', 'request_id': 'req-40aba7c6-08dc-48c0-9592-34bb1733a46e'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:24.026467 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-40aba7c6-08dc-48c0-9592-34bb1733a46e admin admin] The token cache is being invalidate because project f99eef8b6cc64ee8aae548862078c315 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:48:24.026907 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-40aba7c6-08dc-48c0-9592-34bb1733a46e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:24.027639 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-40aba7c6-08dc-48c0-9592-34bb1733a46e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:24.029407 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1284/2546] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:48:23 2022] DELETE /identity/v3/projects/f99eef8b6cc64ee8aae548862078c315 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:24.033514 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379858a3-6818-4fe6-b48d-e9e248c38123 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:24.034969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-379858a3-6818-4fe6-b48d-e9e248c38123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:24.051392 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379858a3-6818-4fe6-b48d-e9e248c38123 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:24.115194 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc77819-0334-4ba0-999f-31b494d22056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:24.115856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc77819-0334-4ba0-999f-31b494d22056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:24.116146 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc77819-0334-4ba0-999f-31b494d22056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:24.116427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc77819-0334-4ba0-999f-31b494d22056 admin admin] PATH_INFO: `/v3/users/0870d660b03745dc9cba85b0f729fdb6` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:24.117653 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc77819-0334-4ba0-999f-31b494d22056 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0870d660b03745dc9cba85b0f729fdb6)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:24.120497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc77819-0334-4ba0-999f-31b494d22056 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:24.122830 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:24.169349 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cbc77819-0334-4ba0-999f-31b494d22056 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0870d660b03745dc9cba85b0f729fdb6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:24.176243 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cbc77819-0334-4ba0-999f-31b494d22056 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0870d660b03745dc9cba85b0f729fdb6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:24.176917 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cbc77819-0334-4ba0-999f-31b494d22056 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0870d660b03745dc9cba85b0f729fdb6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:24.185361 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cbc77819-0334-4ba0-999f-31b494d22056 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0870d660b03745dc9cba85b0f729fdb6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:24.186021 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cbc77819-0334-4ba0-999f-31b494d22056 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0870d660b03745dc9cba85b0f729fdb6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:24.203905 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1263/2547] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:48:24 2022] DELETE /identity/v3/users/0870d660b03745dc9cba85b0f729fdb6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:24.394755 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aba7c6-08dc-48c0-9592-34bb1733a46e admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:24.396219 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-40aba7c6-08dc-48c0-9592-34bb1733a46e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:24.411980 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aba7c6-08dc-48c0-9592-34bb1733a46e admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:24.473694 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc429b78-ab3e-4b4d-bd3b-bbe3dbd9074a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:24.474329 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc429b78-ab3e-4b4d-bd3b-bbe3dbd9074a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:24.474606 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc429b78-ab3e-4b4d-bd3b-bbe3dbd9074a admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:24.474875 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc429b78-ab3e-4b4d-bd3b-bbe3dbd9074a admin admin] PATH_INFO: `/v3/projects/79614c93071c4352a501b289e4609ba4` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:24.476006 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc429b78-ab3e-4b4d-bd3b-bbe3dbd9074a admin admin] RBAC: Authorizing `identity:delete_project(project_id=79614c93071c4352a501b289e4609ba4)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:24.477943 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc429b78-ab3e-4b4d-bd3b-bbe3dbd9074a admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:24.489692 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:24.533296 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dc429b78-ab3e-4b4d-bd3b-bbe3dbd9074a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79614c93071c4352a501b289e4609ba4', 'request_id': 'req-dc429b78-ab3e-4b4d-bd3b-bbe3dbd9074a'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:24.540684 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dc429b78-ab3e-4b4d-bd3b-bbe3dbd9074a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79614c93071c4352a501b289e4609ba4', 'request_id': 'req-dc429b78-ab3e-4b4d-bd3b-bbe3dbd9074a'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:24.548726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dc429b78-ab3e-4b4d-bd3b-bbe3dbd9074a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79614c93071c4352a501b289e4609ba4', 'request_id': 'req-dc429b78-ab3e-4b4d-bd3b-bbe3dbd9074a'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:24.560170 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dc429b78-ab3e-4b4d-bd3b-bbe3dbd9074a admin admin] The token cache is being invalidate because project 79614c93071c4352a501b289e4609ba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:48:24.560461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dc429b78-ab3e-4b4d-bd3b-bbe3dbd9074a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:24.561099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-dc429b78-ab3e-4b4d-bd3b-bbe3dbd9074a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:24.562633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1285/2548] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:48:24 2022] DELETE /identity/v3/projects/79614c93071c4352a501b289e4609ba4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:24.567521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f760b6-480e-44d2-b36c-f447c562fb31 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:24.567999 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f760b6-480e-44d2-b36c-f447c562fb31 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:24.568106 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f760b6-480e-44d2-b36c-f447c562fb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:24.587535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d1f760b6-480e-44d2-b36c-f447c562fb31 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:24.588908 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d1f760b6-480e-44d2-b36c-f447c562fb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:24.651262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1264/2549] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:24.660062 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b62d0a-48bf-4e77-9212-c174a3845144 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:24.660365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b62d0a-48bf-4e77-9212-c174a3845144 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:24.660669 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b62d0a-48bf-4e77-9212-c174a3845144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:24.681093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-b4b62d0a-48bf-4e77-9212-c174a3845144 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:24.682936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b4b62d0a-48bf-4e77-9212-c174a3845144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:24.690851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1286/2550] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:24.695074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f760b6-480e-44d2-b36c-f447c562fb31 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:24.705177 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f760b6-480e-44d2-b36c-f447c562fb31 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:24.709698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f954a-4699-4e94-9080-2c59e34eb7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:24.710307 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f954a-4699-4e94-9080-2c59e34eb7e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:24.710588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f954a-4699-4e94-9080-2c59e34eb7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:24.710862 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f954a-4699-4e94-9080-2c59e34eb7e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:24.711570 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f954a-4699-4e94-9080-2c59e34eb7e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:48:24.711876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f954a-4699-4e94-9080-2c59e34eb7e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:24.713492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f954a-4699-4e94-9080-2c59e34eb7e6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:24.724866 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1265/2551] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:48:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:24.729595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b62d0a-48bf-4e77-9212-c174a3845144 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:24.738991 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b62d0a-48bf-4e77-9212-c174a3845144 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:24.743727 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82b44d-a07a-4d0d-a2c5-e4258d2a8ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:24.745361 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82b44d-a07a-4d0d-a2c5-e4258d2a8ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:24.745641 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82b44d-a07a-4d0d-a2c5-e4258d2a8ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:24.745911 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82b44d-a07a-4d0d-a2c5-e4258d2a8ca6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:24.746563 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82b44d-a07a-4d0d-a2c5-e4258d2a8ca6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:24.748452 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82b44d-a07a-4d0d-a2c5-e4258d2a8ca6 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:24.777819 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1287/2552] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:48:24 2022] POST /identity/v3/projects => generated 376 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:24.781942 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f954a-4699-4e94-9080-2c59e34eb7e6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:24.792995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f954a-4699-4e94-9080-2c59e34eb7e6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:24.849681 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba01b8c-3fc2-4ed7-a92e-df11789d1945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:24.850304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba01b8c-3fc2-4ed7-a92e-df11789d1945 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:24.850590 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba01b8c-3fc2-4ed7-a92e-df11789d1945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:24.850870 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba01b8c-3fc2-4ed7-a92e-df11789d1945 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:24.851534 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba01b8c-3fc2-4ed7-a92e-df11789d1945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:24.854037 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba01b8c-3fc2-4ed7-a92e-df11789d1945 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:24.870122 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:24.877003 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1266/2553] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:48:24 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:48:24.881383 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82b44d-a07a-4d0d-a2c5-e4258d2a8ca6 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:24.893673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82b44d-a07a-4d0d-a2c5-e4258d2a8ca6 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:24.898638 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e56baa-7ab5-454e-a024-6a04867974f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:24.899256 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e56baa-7ab5-454e-a024-6a04867974f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:24.899540 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e56baa-7ab5-454e-a024-6a04867974f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:24.899815 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e56baa-7ab5-454e-a024-6a04867974f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:24.901001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e56baa-7ab5-454e-a024-6a04867974f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:24.906471 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e56baa-7ab5-454e-a024-6a04867974f4 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:24.913042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1288/2554] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:24 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:24.918006 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba01b8c-3fc2-4ed7-a92e-df11789d1945 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:24.928497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba01b8c-3fc2-4ed7-a92e-df11789d1945 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:24.934221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16692cd9-9ea2-4408-8fd9-0d38adb9034e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:24.934682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16692cd9-9ea2-4408-8fd9-0d38adb9034e admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:24.934967 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16692cd9-9ea2-4408-8fd9-0d38adb9034e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:24.935248 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16692cd9-9ea2-4408-8fd9-0d38adb9034e admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:24.935977 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16692cd9-9ea2-4408-8fd9-0d38adb9034e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:24.937608 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16692cd9-9ea2-4408-8fd9-0d38adb9034e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:24.950997 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1267/2555] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:24 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:24.955332 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e56baa-7ab5-454e-a024-6a04867974f4 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:24.966401 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e56baa-7ab5-454e-a024-6a04867974f4 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:24.973023 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a975966-1fe5-4a7b-aff1-b2daed729037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:24.973744 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a975966-1fe5-4a7b-aff1-b2daed729037 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:24.974008 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a975966-1fe5-4a7b-aff1-b2daed729037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:24.974288 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a975966-1fe5-4a7b-aff1-b2daed729037 admin admin] PATH_INFO: `/v3/projects/ad8471e7861f4d9f8e35aa814c4e4b54/users/2af59a680f2f49908a10520dea313dfe/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:24.993181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a975966-1fe5-4a7b-aff1-b2daed729037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8471e7861f4d9f8e35aa814c4e4b54, user_id=2af59a680f2f49908a10520dea313dfe, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:24.995992 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a975966-1fe5-4a7b-aff1-b2daed729037 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:25.021562 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1289/2556] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:24 2022] PUT /identity/v3/projects/ad8471e7861f4d9f8e35aa814c4e4b54/users/2af59a680f2f49908a10520dea313dfe/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:25.023036 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee796d4-9d36-4170-a3d3-ae5dcbd74d60 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:25.023541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee796d4-9d36-4170-a3d3-ae5dcbd74d60 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:25.023908 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee796d4-9d36-4170-a3d3-ae5dcbd74d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:25.054755 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-4ee796d4-9d36-4170-a3d3-ae5dcbd74d60 None None] MFA Rules not processed for user `2af59a680f2f49908a10520dea313dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:25.055885 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4ee796d4-9d36-4170-a3d3-ae5dcbd74d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:25.119969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1268/2557] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:25.124474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a975966-1fe5-4a7b-aff1-b2daed729037 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:25.149078 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee796d4-9d36-4170-a3d3-ae5dcbd74d60 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:25.151199 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4ee796d4-9d36-4170-a3d3-ae5dcbd74d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:25.191112 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a975966-1fe5-4a7b-aff1-b2daed729037 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:25.196216 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05fa9a-5cc1-4276-8a9c-3e43e93dfe09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:25.196830 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05fa9a-5cc1-4276-8a9c-3e43e93dfe09 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:25.197665 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05fa9a-5cc1-4276-8a9c-3e43e93dfe09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:25.197948 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05fa9a-5cc1-4276-8a9c-3e43e93dfe09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:25.198790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05fa9a-5cc1-4276-8a9c-3e43e93dfe09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:25.200711 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05fa9a-5cc1-4276-8a9c-3e43e93dfe09 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:25.218029 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee796d4-9d36-4170-a3d3-ae5dcbd74d60 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:25.223413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1ebfb-c483-47ea-ab6d-85b7a79486d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:25.224100 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1ebfb-c483-47ea-ab6d-85b7a79486d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:25.224386 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1ebfb-c483-47ea-ab6d-85b7a79486d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:25.224666 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1ebfb-c483-47ea-ab6d-85b7a79486d3 admin admin] PATH_INFO: `/v3/users/7c91ac47ab3a4a0e90e9d4c3a064237e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:25.225895 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae1ebfb-c483-47ea-ab6d-85b7a79486d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c91ac47ab3a4a0e90e9d4c3a064237e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:25.227873 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae1ebfb-c483-47ea-ab6d-85b7a79486d3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:25.228025 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1290/2558] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:48:25 2022] POST /identity/v3/projects => generated 374 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:25.232086 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05fa9a-5cc1-4276-8a9c-3e43e93dfe09 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:25.260937 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:25.282134 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cae1ebfb-c483-47ea-ab6d-85b7a79486d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c91ac47ab3a4a0e90e9d4c3a064237e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:25.289936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cae1ebfb-c483-47ea-ab6d-85b7a79486d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c91ac47ab3a4a0e90e9d4c3a064237e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:25.290663 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cae1ebfb-c483-47ea-ab6d-85b7a79486d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c91ac47ab3a4a0e90e9d4c3a064237e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:25.298974 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cae1ebfb-c483-47ea-ab6d-85b7a79486d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c91ac47ab3a4a0e90e9d4c3a064237e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:25.299840 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cae1ebfb-c483-47ea-ab6d-85b7a79486d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c91ac47ab3a4a0e90e9d4c3a064237e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:25.305880 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05fa9a-5cc1-4276-8a9c-3e43e93dfe09 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:25.310601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afbd2b-7149-4548-9b7b-d771e434b92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:25.311210 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afbd2b-7149-4548-9b7b-d771e434b92c admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:25.311481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afbd2b-7149-4548-9b7b-d771e434b92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:25.311751 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afbd2b-7149-4548-9b7b-d771e434b92c admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:25.312405 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01afbd2b-7149-4548-9b7b-d771e434b92c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:25.312542 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:25.314474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01afbd2b-7149-4548-9b7b-d771e434b92c admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:25.321803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1269/2559] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:48:25 2022] DELETE /identity/v3/users/7c91ac47ab3a4a0e90e9d4c3a064237e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:48:25.326648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b13b4b-607e-472a-91de-99f82dfb1863 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:25.326939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b13b4b-607e-472a-91de-99f82dfb1863 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:25.327669 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b13b4b-607e-472a-91de-99f82dfb1863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:25.341673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1291/2560] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:48:25 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:48:25.345457 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afbd2b-7149-4548-9b7b-d771e434b92c admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:25.347090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-01afbd2b-7149-4548-9b7b-d771e434b92c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:25.350497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-41b13b4b-607e-472a-91de-99f82dfb1863 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:25.351665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-41b13b4b-607e-472a-91de-99f82dfb1863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:25.359284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afbd2b-7149-4548-9b7b-d771e434b92c admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:25.414269 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1270/2561] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:48:25.417217 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dd34c5-3c5c-4914-a718-71d0f846a0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:25.417932 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd34c5-3c5c-4914-a718-71d0f846a0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:25.418660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd34c5-3c5c-4914-a718-71d0f846a0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:25.418660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd34c5-3c5c-4914-a718-71d0f846a0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:25.419141 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd34c5-3c5c-4914-a718-71d0f846a0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:25.420736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd34c5-3c5c-4914-a718-71d0f846a0e6 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:25.425593 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b13b4b-607e-472a-91de-99f82dfb1863 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:25.427443 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-41b13b4b-607e-472a-91de-99f82dfb1863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:25.431045 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1292/2562] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:25 2022] GET /identity/v3/roles => generated 1427 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:25.435456 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dd34c5-3c5c-4914-a718-71d0f846a0e6 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:25.445064 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dd34c5-3c5c-4914-a718-71d0f846a0e6 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:25.450661 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab5149-d0a4-446a-b375-815b3269d1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:25.451314 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab5149-d0a4-446a-b375-815b3269d1d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:25.451599 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab5149-d0a4-446a-b375-815b3269d1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:25.451904 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab5149-d0a4-446a-b375-815b3269d1d3 admin admin] PATH_INFO: `/v3/projects/a9425488d02948f0b4561020f3db36d5/users/2a4326c349624d04b7a29ca4dbff1e0c/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:25.467476 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab5149-d0a4-446a-b375-815b3269d1d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9425488d02948f0b4561020f3db36d5, user_id=2a4326c349624d04b7a29ca4dbff1e0c, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:25.470247 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab5149-d0a4-446a-b375-815b3269d1d3 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:25.484844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1293/2563] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:25 2022] PUT /identity/v3/projects/a9425488d02948f0b4561020f3db36d5/users/2a4326c349624d04b7a29ca4dbff1e0c/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:48:25.488823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab5149-d0a4-446a-b375-815b3269d1d3 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:25.490004 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b13b4b-607e-472a-91de-99f82dfb1863 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:25.541050 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe46df6-bb0e-4080-834e-76e1efc25d3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:25.541638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe46df6-bb0e-4080-834e-76e1efc25d3b service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:25.541920 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe46df6-bb0e-4080-834e-76e1efc25d3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:25.542198 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe46df6-bb0e-4080-834e-76e1efc25d3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:25.551810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab5149-d0a4-446a-b375-815b3269d1d3 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:25.552107 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe46df6-bb0e-4080-834e-76e1efc25d3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:25.553720 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe46df6-bb0e-4080-834e-76e1efc25d3b service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:25.557535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae59136-a5b9-4a8e-87fb-b4507f39a08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:25.558084 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae59136-a5b9-4a8e-87fb-b4507f39a08d admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:25.558365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae59136-a5b9-4a8e-87fb-b4507f39a08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:25.558636 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae59136-a5b9-4a8e-87fb-b4507f39a08d admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:25.559317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae59136-a5b9-4a8e-87fb-b4507f39a08d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:25.560893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae59136-a5b9-4a8e-87fb-b4507f39a08d admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:25.565079 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1271/2564] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:25 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:48:25.570997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1294/2565] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:25 2022] GET /identity/v3/roles => generated 1427 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:48:25.575615 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe46df6-bb0e-4080-834e-76e1efc25d3b service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:25.585660 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe46df6-bb0e-4080-834e-76e1efc25d3b service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:25.590832 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcfe0a-2274-415e-b25d-d293e12c6ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:25.591499 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcfe0a-2274-415e-b25d-d293e12c6ced admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:25.591782 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcfe0a-2274-415e-b25d-d293e12c6ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:25.592059 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcfe0a-2274-415e-b25d-d293e12c6ced admin admin] PATH_INFO: `/v3/projects/a9425488d02948f0b4561020f3db36d5/users/2a4326c349624d04b7a29ca4dbff1e0c/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:25.594101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fcfe0a-2274-415e-b25d-d293e12c6ced admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9425488d02948f0b4561020f3db36d5, user_id=2a4326c349624d04b7a29ca4dbff1e0c, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:25.596742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fcfe0a-2274-415e-b25d-d293e12c6ced admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:25.608213 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1272/2566] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:25 2022] PUT /identity/v3/projects/a9425488d02948f0b4561020f3db36d5/users/2a4326c349624d04b7a29ca4dbff1e0c/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:48:25.619694 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bc892-90e9-4c25-8052-cd60fbdeffc8 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:25.620029 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bc892-90e9-4c25-8052-cd60fbdeffc8 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:25.620365 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bc892-90e9-4c25-8052-cd60fbdeffc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:25.650515 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-8e0bc892-90e9-4c25-8052-cd60fbdeffc8 None None] MFA Rules not processed for user `2a4326c349624d04b7a29ca4dbff1e0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:25.651760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-8e0bc892-90e9-4c25-8052-cd60fbdeffc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:25.660721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:25.725048 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1295/2567] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:25.735978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcfe0a-2274-415e-b25d-d293e12c6ced admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:25.737824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-85fcfe0a-2274-415e-b25d-d293e12c6ced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:25.788645 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:25.802405 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcfe0a-2274-415e-b25d-d293e12c6ced admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:25.806181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f3831-d4c6-4a9b-9f6e-b3356282a1b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:25.807684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f3831-d4c6-4a9b-9f6e-b3356282a1b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:25.808010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f3831-d4c6-4a9b-9f6e-b3356282a1b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:25.808286 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f3831-d4c6-4a9b-9f6e-b3356282a1b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:25.810497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8e6f3831-d4c6-4a9b-9f6e-b3356282a1b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:25.862119 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bc892-90e9-4c25-8052-cd60fbdeffc8 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:25.864808 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-8e0bc892-90e9-4c25-8052-cd60fbdeffc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:25.887637 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f3831-d4c6-4a9b-9f6e-b3356282a1b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:25.889105 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f3831-d4c6-4a9b-9f6e-b3356282a1b1 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:25.900714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1273/2568] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:48:25.941789 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bc892-90e9-4c25-8052-cd60fbdeffc8 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:25.946480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96225d-4a4a-4347-a1ec-8cfa09e5feaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:25.947330 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96225d-4a4a-4347-a1ec-8cfa09e5feaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:25.947770 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96225d-4a4a-4347-a1ec-8cfa09e5feaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:25.948052 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96225d-4a4a-4347-a1ec-8cfa09e5feaa admin admin] PATH_INFO: `/v3/projects/a8f6d1f35f45461d8f0fada0377a40ec` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:25.949456 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c96225d-4a4a-4347-a1ec-8cfa09e5feaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8f6d1f35f45461d8f0fada0377a40ec)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:25.951392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c96225d-4a4a-4347-a1ec-8cfa09e5feaa admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:26.022818 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5c96225d-4a4a-4347-a1ec-8cfa09e5feaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8f6d1f35f45461d8f0fada0377a40ec', 'request_id': 'req-5c96225d-4a4a-4347-a1ec-8cfa09e5feaa'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:26.032251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5c96225d-4a4a-4347-a1ec-8cfa09e5feaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8f6d1f35f45461d8f0fada0377a40ec', 'request_id': 'req-5c96225d-4a4a-4347-a1ec-8cfa09e5feaa'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:26.042773 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5c96225d-4a4a-4347-a1ec-8cfa09e5feaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8f6d1f35f45461d8f0fada0377a40ec', 'request_id': 'req-5c96225d-4a4a-4347-a1ec-8cfa09e5feaa'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:26.055453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5c96225d-4a4a-4347-a1ec-8cfa09e5feaa admin admin] The token cache is being invalidate because project a8f6d1f35f45461d8f0fada0377a40ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:48:26.055876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5c96225d-4a4a-4347-a1ec-8cfa09e5feaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:26.057807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-5c96225d-4a4a-4347-a1ec-8cfa09e5feaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:26.060455 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1296/2569] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:48:25 2022] DELETE /identity/v3/projects/a8f6d1f35f45461d8f0fada0377a40ec => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:48:26.065334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f3831-d4c6-4a9b-9f6e-b3356282a1b1 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:26.085446 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f3831-d4c6-4a9b-9f6e-b3356282a1b1 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:26.153870 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c517c2-aa3d-448f-811b-ce76c5c2d337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:26.154594 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c517c2-aa3d-448f-811b-ce76c5c2d337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:26.154872 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c517c2-aa3d-448f-811b-ce76c5c2d337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:26.155162 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c517c2-aa3d-448f-811b-ce76c5c2d337 admin admin] PATH_INFO: `/v3/users/a1f74cfb72554635811f0253e3cb94a6` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:26.156524 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c517c2-aa3d-448f-811b-ce76c5c2d337 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1f74cfb72554635811f0253e3cb94a6)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:26.158887 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c517c2-aa3d-448f-811b-ce76c5c2d337 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:26.213453 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-65c517c2-aa3d-448f-811b-ce76c5c2d337 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1f74cfb72554635811f0253e3cb94a6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:26.221382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-65c517c2-aa3d-448f-811b-ce76c5c2d337 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1f74cfb72554635811f0253e3cb94a6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:26.222223 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-65c517c2-aa3d-448f-811b-ce76c5c2d337 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1f74cfb72554635811f0253e3cb94a6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:26.236190 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-65c517c2-aa3d-448f-811b-ce76c5c2d337 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1f74cfb72554635811f0253e3cb94a6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:26.237303 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-65c517c2-aa3d-448f-811b-ce76c5c2d337 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1f74cfb72554635811f0253e3cb94a6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:26.257692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1274/2570] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:48:26 2022] DELETE /identity/v3/users/a1f74cfb72554635811f0253e3cb94a6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:48:26.534671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96225d-4a4a-4347-a1ec-8cfa09e5feaa admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:26.536132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-5c96225d-4a4a-4347-a1ec-8cfa09e5feaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:26.561703 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96225d-4a4a-4347-a1ec-8cfa09e5feaa admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:26.596795 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:26.623806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e8ea3-815e-46e3-b7ae-de7815446c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:26.624453 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e8ea3-815e-46e3-b7ae-de7815446c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:26.624738 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e8ea3-815e-46e3-b7ae-de7815446c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:26.625016 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e8ea3-815e-46e3-b7ae-de7815446c30 admin admin] PATH_INFO: `/v3/projects/203f1f1a102541bf9102af33f4d50cdb` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:26.626211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e8ea3-815e-46e3-b7ae-de7815446c30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=203f1f1a102541bf9102af33f4d50cdb)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:26.628112 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e8ea3-815e-46e3-b7ae-de7815446c30 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:26.675290 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-792e8ea3-815e-46e3-b7ae-de7815446c30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '203f1f1a102541bf9102af33f4d50cdb', 'request_id': 'req-792e8ea3-815e-46e3-b7ae-de7815446c30'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:26.682188 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-792e8ea3-815e-46e3-b7ae-de7815446c30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '203f1f1a102541bf9102af33f4d50cdb', 'request_id': 'req-792e8ea3-815e-46e3-b7ae-de7815446c30'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:26.690843 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-792e8ea3-815e-46e3-b7ae-de7815446c30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '203f1f1a102541bf9102af33f4d50cdb', 'request_id': 'req-792e8ea3-815e-46e3-b7ae-de7815446c30'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:26.702510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-792e8ea3-815e-46e3-b7ae-de7815446c30 admin admin] The token cache is being invalidate because project 203f1f1a102541bf9102af33f4d50cdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:48:26.702800 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-792e8ea3-815e-46e3-b7ae-de7815446c30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:26.703423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-792e8ea3-815e-46e3-b7ae-de7815446c30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:26.704984 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1297/2571] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:48:26 2022] DELETE /identity/v3/projects/203f1f1a102541bf9102af33f4d50cdb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:26.711065 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810ddd0-ca18-417d-8f49-ebc782092f19 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:26.711551 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810ddd0-ca18-417d-8f49-ebc782092f19 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:26.711655 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810ddd0-ca18-417d-8f49-ebc782092f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:26.732195 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a810ddd0-ca18-417d-8f49-ebc782092f19 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:26.733438 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a810ddd0-ca18-417d-8f49-ebc782092f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:26.742877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:26.797041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1275/2572] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:26.805402 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c35c63-9830-4e00-8d7b-bc89c292ec1c None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:26.805685 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c35c63-9830-4e00-8d7b-bc89c292ec1c None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:26.805957 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c35c63-9830-4e00-8d7b-bc89c292ec1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:26.826367 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-d6c35c63-9830-4e00-8d7b-bc89c292ec1c None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:26.827568 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d6c35c63-9830-4e00-8d7b-bc89c292ec1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:26.838128 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1298/2573] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:26.842545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810ddd0-ca18-417d-8f49-ebc782092f19 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:26.852710 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810ddd0-ca18-417d-8f49-ebc782092f19 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:26.857820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3cb80-db78-488a-9408-bf074e743606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:26.858441 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3cb80-db78-488a-9408-bf074e743606 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:26.858727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3cb80-db78-488a-9408-bf074e743606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:26.859004 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3cb80-db78-488a-9408-bf074e743606 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:26.859703 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3cb80-db78-488a-9408-bf074e743606 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:48:26.860008 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3cb80-db78-488a-9408-bf074e743606 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:26.861635 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3cb80-db78-488a-9408-bf074e743606 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:26.873853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1276/2574] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:48:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:26.878375 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c35c63-9830-4e00-8d7b-bc89c292ec1c None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:26.888401 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c35c63-9830-4e00-8d7b-bc89c292ec1c None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:26.893063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e6194-e958-4a41-a07c-618ed7ebd72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:26.893698 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e6194-e958-4a41-a07c-618ed7ebd72f admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:26.893982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e6194-e958-4a41-a07c-618ed7ebd72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:26.894262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e6194-e958-4a41-a07c-618ed7ebd72f admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:26.894955 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e6194-e958-4a41-a07c-618ed7ebd72f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:26.896841 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e6194-e958-4a41-a07c-618ed7ebd72f admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:26.926793 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1299/2575] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:48:26 2022] POST /identity/v3/projects => generated 372 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:26.931145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3cb80-db78-488a-9408-bf074e743606 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:26.942180 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3cb80-db78-488a-9408-bf074e743606 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:26.999321 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93ae0c-309e-47f4-9e29-3faba0bbf46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:26.999939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93ae0c-309e-47f4-9e29-3faba0bbf46b admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:27.000223 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93ae0c-309e-47f4-9e29-3faba0bbf46b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:27.000500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93ae0c-309e-47f4-9e29-3faba0bbf46b admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:27.001216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93ae0c-309e-47f4-9e29-3faba0bbf46b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:27.003137 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93ae0c-309e-47f4-9e29-3faba0bbf46b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:27.024570 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1277/2576] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:48:26 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:48:27.029260 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e6194-e958-4a41-a07c-618ed7ebd72f admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:27.039672 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e6194-e958-4a41-a07c-618ed7ebd72f admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:27.046211 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ea63e-3c44-43f2-9e18-91f2009d9250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:27.047060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ea63e-3c44-43f2-9e18-91f2009d9250 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:27.047506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ea63e-3c44-43f2-9e18-91f2009d9250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:27.047904 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ea63e-3c44-43f2-9e18-91f2009d9250 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:27.048776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ea63e-3c44-43f2-9e18-91f2009d9250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:27.050573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ea63e-3c44-43f2-9e18-91f2009d9250 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:27.061474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1300/2577] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:27 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:27.065717 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93ae0c-309e-47f4-9e29-3faba0bbf46b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:27.076008 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93ae0c-309e-47f4-9e29-3faba0bbf46b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:27.080765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6855d171-d21f-484a-9e68-dbd254fc4814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:27.081424 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855d171-d21f-484a-9e68-dbd254fc4814 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:27.081713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855d171-d21f-484a-9e68-dbd254fc4814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:27.081992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855d171-d21f-484a-9e68-dbd254fc4814 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:27.082691 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6855d171-d21f-484a-9e68-dbd254fc4814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:27.084287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6855d171-d21f-484a-9e68-dbd254fc4814 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:27.096101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1278/2578] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:27 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:27.100329 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ea63e-3c44-43f2-9e18-91f2009d9250 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:27.112542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ea63e-3c44-43f2-9e18-91f2009d9250 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:27.117295 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a65ef-89fe-4461-a925-74eca4441466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:27.117967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a65ef-89fe-4461-a925-74eca4441466 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:27.118253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a65ef-89fe-4461-a925-74eca4441466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:27.118532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a65ef-89fe-4461-a925-74eca4441466 admin admin] PATH_INFO: `/v3/projects/8b4ccc78886e49bab441df974a59f153/users/cc2816fdac31480e83d01d6733136944/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:27.135559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a65ef-89fe-4461-a925-74eca4441466 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b4ccc78886e49bab441df974a59f153, user_id=cc2816fdac31480e83d01d6733136944, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:27.138298 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a65ef-89fe-4461-a925-74eca4441466 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:27.155098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1301/2579] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:27 2022] PUT /identity/v3/projects/8b4ccc78886e49bab441df974a59f153/users/cc2816fdac31480e83d01d6733136944/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:48:27.167111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e34235-92d0-4997-90f3-b685f65eec83 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:27.167412 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e34235-92d0-4997-90f3-b685f65eec83 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:27.167826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e34235-92d0-4997-90f3-b685f65eec83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:27.195084 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c9e34235-92d0-4997-90f3-b685f65eec83 None None] MFA Rules not processed for user `cc2816fdac31480e83d01d6733136944`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:27.196294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c9e34235-92d0-4997-90f3-b685f65eec83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:27.261743 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1279/2580] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:27.272882 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a65ef-89fe-4461-a925-74eca4441466 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:27.274834 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-0a7a65ef-89fe-4461-a925-74eca4441466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:27.334018 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a65ef-89fe-4461-a925-74eca4441466 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:27.338351 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364ae37-4582-4380-be54-ca4c0170856b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:27.339082 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364ae37-4582-4380-be54-ca4c0170856b service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:27.339387 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364ae37-4582-4380-be54-ca4c0170856b service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:27.339657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364ae37-4582-4380-be54-ca4c0170856b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:27.349246 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364ae37-4582-4380-be54-ca4c0170856b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:27.350681 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364ae37-4582-4380-be54-ca4c0170856b service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:27.360979 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1302/2581] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:48:28.481568 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e34235-92d0-4997-90f3-b685f65eec83 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:28.483019 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c9e34235-92d0-4997-90f3-b685f65eec83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:28.496953 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:28.551594 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e34235-92d0-4997-90f3-b685f65eec83 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:28.556156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbde2e4-41cb-4cba-808d-bc568282c950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:28.557026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbde2e4-41cb-4cba-808d-bc568282c950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:28.557333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbde2e4-41cb-4cba-808d-bc568282c950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:28.557617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbde2e4-41cb-4cba-808d-bc568282c950 admin admin] PATH_INFO: `/v3/users/2af59a680f2f49908a10520dea313dfe` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:28.558809 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbde2e4-41cb-4cba-808d-bc568282c950 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2af59a680f2f49908a10520dea313dfe)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:28.560732 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbde2e4-41cb-4cba-808d-bc568282c950 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:28.608529 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1dbde2e4-41cb-4cba-808d-bc568282c950 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2af59a680f2f49908a10520dea313dfe'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:28.615442 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1dbde2e4-41cb-4cba-808d-bc568282c950 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2af59a680f2f49908a10520dea313dfe'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:28.616102 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1dbde2e4-41cb-4cba-808d-bc568282c950 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2af59a680f2f49908a10520dea313dfe'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:28.623711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1dbde2e4-41cb-4cba-808d-bc568282c950 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2af59a680f2f49908a10520dea313dfe'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:28.624324 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1dbde2e4-41cb-4cba-808d-bc568282c950 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2af59a680f2f49908a10520dea313dfe'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:28.641860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1280/2582] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:48:28 2022] DELETE /identity/v3/users/2af59a680f2f49908a10520dea313dfe => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:28.646600 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1a4d9-3003-4211-8bb2-b47ca1b3b296 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:28.647157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1a4d9-3003-4211-8bb2-b47ca1b3b296 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:28.647495 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1a4d9-3003-4211-8bb2-b47ca1b3b296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:28.656021 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:28.668706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-b5a1a4d9-3003-4211-8bb2-b47ca1b3b296 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:28.670197 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b5a1a4d9-3003-4211-8bb2-b47ca1b3b296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:28.731054 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1303/2583] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:28.740892 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbde2e4-41cb-4cba-808d-bc568282c950 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:28.742601 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1dbde2e4-41cb-4cba-808d-bc568282c950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:28.805461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbde2e4-41cb-4cba-808d-bc568282c950 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:28.809564 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d5c71-b653-40f2-9414-49e9016160de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:28.810196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d5c71-b653-40f2-9414-49e9016160de service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:28.810484 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d5c71-b653-40f2-9414-49e9016160de service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:28.810791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d5c71-b653-40f2-9414-49e9016160de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:28.820612 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d5c71-b653-40f2-9414-49e9016160de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:28.822849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d5c71-b653-40f2-9414-49e9016160de service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:28.834597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1281/2584] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:48:29.322690 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1a4d9-3003-4211-8bb2-b47ca1b3b296 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:29.324277 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b5a1a4d9-3003-4211-8bb2-b47ca1b3b296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:29.337273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1a4d9-3003-4211-8bb2-b47ca1b3b296 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:29.343038 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9171d8b-e5ac-4f02-b06b-6024b04dbe5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:29.343751 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9171d8b-e5ac-4f02-b06b-6024b04dbe5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:29.344135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9171d8b-e5ac-4f02-b06b-6024b04dbe5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:29.344663 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9171d8b-e5ac-4f02-b06b-6024b04dbe5b admin admin] PATH_INFO: `/v3/projects/ad8471e7861f4d9f8e35aa814c4e4b54` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:29.347397 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9171d8b-e5ac-4f02-b06b-6024b04dbe5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad8471e7861f4d9f8e35aa814c4e4b54)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:29.349912 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9171d8b-e5ac-4f02-b06b-6024b04dbe5b admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:29.411242 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b9171d8b-e5ac-4f02-b06b-6024b04dbe5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad8471e7861f4d9f8e35aa814c4e4b54', 'request_id': 'req-b9171d8b-e5ac-4f02-b06b-6024b04dbe5b'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:29.419074 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b9171d8b-e5ac-4f02-b06b-6024b04dbe5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad8471e7861f4d9f8e35aa814c4e4b54', 'request_id': 'req-b9171d8b-e5ac-4f02-b06b-6024b04dbe5b'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:29.427973 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b9171d8b-e5ac-4f02-b06b-6024b04dbe5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad8471e7861f4d9f8e35aa814c4e4b54', 'request_id': 'req-b9171d8b-e5ac-4f02-b06b-6024b04dbe5b'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:29.440725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b9171d8b-e5ac-4f02-b06b-6024b04dbe5b admin admin] The token cache is being invalidate because project ad8471e7861f4d9f8e35aa814c4e4b54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:48:29.441061 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b9171d8b-e5ac-4f02-b06b-6024b04dbe5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:29.442587 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b9171d8b-e5ac-4f02-b06b-6024b04dbe5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:29.444503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1304/2585] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:48:29 2022] DELETE /identity/v3/projects/ad8471e7861f4d9f8e35aa814c4e4b54 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:29.450451 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d5c71-b653-40f2-9414-49e9016160de service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:29.461363 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d5c71-b653-40f2-9414-49e9016160de service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:29.498374 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:29.535448 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4ab9e-2bbb-4194-8b2f-add9ba48b28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:29.536110 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4ab9e-2bbb-4194-8b2f-add9ba48b28c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:29.536393 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4ab9e-2bbb-4194-8b2f-add9ba48b28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:29.536665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4ab9e-2bbb-4194-8b2f-add9ba48b28c admin admin] PATH_INFO: `/v3/users/2a4326c349624d04b7a29ca4dbff1e0c` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:29.537863 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4ab9e-2bbb-4194-8b2f-add9ba48b28c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a4326c349624d04b7a29ca4dbff1e0c)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:29.539770 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4ab9e-2bbb-4194-8b2f-add9ba48b28c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:29.591477 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-66f4ab9e-2bbb-4194-8b2f-add9ba48b28c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a4326c349624d04b7a29ca4dbff1e0c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:29.598911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-66f4ab9e-2bbb-4194-8b2f-add9ba48b28c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a4326c349624d04b7a29ca4dbff1e0c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:29.599640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-66f4ab9e-2bbb-4194-8b2f-add9ba48b28c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a4326c349624d04b7a29ca4dbff1e0c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:29.607923 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-66f4ab9e-2bbb-4194-8b2f-add9ba48b28c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a4326c349624d04b7a29ca4dbff1e0c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:29.608739 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-66f4ab9e-2bbb-4194-8b2f-add9ba48b28c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a4326c349624d04b7a29ca4dbff1e0c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:29.627507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1282/2586] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:48:29 2022] DELETE /identity/v3/users/2a4326c349624d04b7a29ca4dbff1e0c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:48:29.924816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9171d8b-e5ac-4f02-b06b-6024b04dbe5b admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:29.926289 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b9171d8b-e5ac-4f02-b06b-6024b04dbe5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:29.939594 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9171d8b-e5ac-4f02-b06b-6024b04dbe5b admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:29.998816 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bbe40-e71b-41c4-ab00-834b5e9b9436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:29.999374 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bbe40-e71b-41c4-ab00-834b5e9b9436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:29.999679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bbe40-e71b-41c4-ab00-834b5e9b9436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:29.999951 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bbe40-e71b-41c4-ab00-834b5e9b9436 admin admin] PATH_INFO: `/v3/projects/a9425488d02948f0b4561020f3db36d5` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:30.001089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9bbe40-e71b-41c4-ab00-834b5e9b9436 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9425488d02948f0b4561020f3db36d5)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:30.003046 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9bbe40-e71b-41c4-ab00-834b5e9b9436 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:30.036870 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:30.051484 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-0f9bbe40-e71b-41c4-ab00-834b5e9b9436 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9425488d02948f0b4561020f3db36d5', 'request_id': 'req-0f9bbe40-e71b-41c4-ab00-834b5e9b9436'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:30.058056 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-0f9bbe40-e71b-41c4-ab00-834b5e9b9436 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9425488d02948f0b4561020f3db36d5', 'request_id': 'req-0f9bbe40-e71b-41c4-ab00-834b5e9b9436'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:30.065637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-0f9bbe40-e71b-41c4-ab00-834b5e9b9436 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9425488d02948f0b4561020f3db36d5', 'request_id': 'req-0f9bbe40-e71b-41c4-ab00-834b5e9b9436'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:30.076867 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-0f9bbe40-e71b-41c4-ab00-834b5e9b9436 admin admin] The token cache is being invalidate because project a9425488d02948f0b4561020f3db36d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:48:30.077192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-0f9bbe40-e71b-41c4-ab00-834b5e9b9436 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:30.077834 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-0f9bbe40-e71b-41c4-ab00-834b5e9b9436 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:30.079355 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1305/2587] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:48:29 2022] DELETE /identity/v3/projects/a9425488d02948f0b4561020f3db36d5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:30.088514 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2222244-e7e4-45ab-a9ed-90ac796ea187 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:30.088865 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2222244-e7e4-45ab-a9ed-90ac796ea187 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:30.089131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2222244-e7e4-45ab-a9ed-90ac796ea187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:30.109052 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f2222244-e7e4-45ab-a9ed-90ac796ea187 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:30.110524 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f2222244-e7e4-45ab-a9ed-90ac796ea187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:30.173823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1283/2588] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:30.183767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a87a3-1b56-4aff-ad94-3097bf02defb None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:30.184054 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a87a3-1b56-4aff-ad94-3097bf02defb None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:30.184335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a87a3-1b56-4aff-ad94-3097bf02defb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:30.204880 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-f16a87a3-1b56-4aff-ad94-3097bf02defb None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:30.206152 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f16a87a3-1b56-4aff-ad94-3097bf02defb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:30.214608 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1306/2589] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:30.218997 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2222244-e7e4-45ab-a9ed-90ac796ea187 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:30.229337 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2222244-e7e4-45ab-a9ed-90ac796ea187 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:30.233911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec5c14-3ad0-478e-9902-696f016ff9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:30.234517 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec5c14-3ad0-478e-9902-696f016ff9af admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:30.234799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec5c14-3ad0-478e-9902-696f016ff9af admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:30.235073 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec5c14-3ad0-478e-9902-696f016ff9af admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:30.235878 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec5c14-3ad0-478e-9902-696f016ff9af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:48:30.236070 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec5c14-3ad0-478e-9902-696f016ff9af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:30.237681 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec5c14-3ad0-478e-9902-696f016ff9af admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:30.250128 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1284/2590] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:48:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:30.254353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a87a3-1b56-4aff-ad94-3097bf02defb None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:30.264369 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a87a3-1b56-4aff-ad94-3097bf02defb None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:30.269199 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c7a22-60ab-4649-b731-2bd20c0e1370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:30.269810 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c7a22-60ab-4649-b731-2bd20c0e1370 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:30.270087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c7a22-60ab-4649-b731-2bd20c0e1370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:30.270358 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c7a22-60ab-4649-b731-2bd20c0e1370 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:30.271008 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c7a22-60ab-4649-b731-2bd20c0e1370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:30.272973 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c7a22-60ab-4649-b731-2bd20c0e1370 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:30.307310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1307/2591] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:48:30 2022] POST /identity/v3/projects => generated 366 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:30.308393 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec5c14-3ad0-478e-9902-696f016ff9af admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:30.318633 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec5c14-3ad0-478e-9902-696f016ff9af admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:30.359848 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:30.382245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9624c-e5d3-4e74-a568-372fb1f62150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:30.382871 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9624c-e5d3-4e74-a568-372fb1f62150 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:30.383153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9624c-e5d3-4e74-a568-372fb1f62150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:30.383428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9624c-e5d3-4e74-a568-372fb1f62150 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:30.384102 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db9624c-e5d3-4e74-a568-372fb1f62150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:30.386735 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db9624c-e5d3-4e74-a568-372fb1f62150 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:30.408624 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1285/2592] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:48:30 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:30.412135 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c7a22-60ab-4649-b731-2bd20c0e1370 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:30.423589 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c7a22-60ab-4649-b731-2bd20c0e1370 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:30.428249 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2582010-0264-45ed-8c8f-d71feee54d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:30.428859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2582010-0264-45ed-8c8f-d71feee54d73 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:30.429173 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2582010-0264-45ed-8c8f-d71feee54d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:30.429451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2582010-0264-45ed-8c8f-d71feee54d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:30.430175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2582010-0264-45ed-8c8f-d71feee54d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:30.432423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2582010-0264-45ed-8c8f-d71feee54d73 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:30.442438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1308/2593] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:30 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:30.446650 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9624c-e5d3-4e74-a568-372fb1f62150 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:30.456964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9624c-e5d3-4e74-a568-372fb1f62150 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:30.462126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87204766-0987-4319-b0a9-f07840edd9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:30.462890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87204766-0987-4319-b0a9-f07840edd9fc admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:30.463054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87204766-0987-4319-b0a9-f07840edd9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:30.463298 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87204766-0987-4319-b0a9-f07840edd9fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:30.463997 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87204766-0987-4319-b0a9-f07840edd9fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:30.465652 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87204766-0987-4319-b0a9-f07840edd9fc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:30.475977 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1286/2594] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:30 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:30.480918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2582010-0264-45ed-8c8f-d71feee54d73 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:30.490746 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2582010-0264-45ed-8c8f-d71feee54d73 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:30.496389 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ad389-df40-4763-8969-3cde2f151b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:30.497030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ad389-df40-4763-8969-3cde2f151b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:30.497370 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ad389-df40-4763-8969-3cde2f151b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:30.497643 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ad389-df40-4763-8969-3cde2f151b50 admin admin] PATH_INFO: `/v3/projects/144fce8b4197412e969dae5a4c545730/users/28171410d16f4a65854540d373ebb3fb/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:30.512413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ad389-df40-4763-8969-3cde2f151b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=144fce8b4197412e969dae5a4c545730, user_id=28171410d16f4a65854540d373ebb3fb, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:30.515071 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ad389-df40-4763-8969-3cde2f151b50 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:30.529450 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1309/2595] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:30 2022] PUT /identity/v3/projects/144fce8b4197412e969dae5a4c545730/users/28171410d16f4a65854540d373ebb3fb/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:30.543425 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46983636-ae4f-41f8-8ddf-36571f213854 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:30.543919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46983636-ae4f-41f8-8ddf-36571f213854 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:30.544033 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46983636-ae4f-41f8-8ddf-36571f213854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:30.551275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ad389-df40-4763-8969-3cde2f151b50 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:30.553004 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ed5ad389-df40-4763-8969-3cde2f151b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:30.575766 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-46983636-ae4f-41f8-8ddf-36571f213854 None None] MFA Rules not processed for user `28171410d16f4a65854540d373ebb3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:30.577136 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-46983636-ae4f-41f8-8ddf-36571f213854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:30.623012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ad389-df40-4763-8969-3cde2f151b50 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:30.627819 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38be73-17a9-4807-95a0-ba1a0488c06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:30.628460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38be73-17a9-4807-95a0-ba1a0488c06b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:30.628830 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38be73-17a9-4807-95a0-ba1a0488c06b admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:30.629129 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38be73-17a9-4807-95a0-ba1a0488c06b admin admin] PATH_INFO: `/v3/users/cc2816fdac31480e83d01d6733136944` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:30.630304 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38be73-17a9-4807-95a0-ba1a0488c06b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc2816fdac31480e83d01d6733136944)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:30.632508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38be73-17a9-4807-95a0-ba1a0488c06b admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:30.644942 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1287/2596] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:30.656479 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46983636-ae4f-41f8-8ddf-36571f213854 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:30.658212 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-46983636-ae4f-41f8-8ddf-36571f213854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:30.681220 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8e38be73-17a9-4807-95a0-ba1a0488c06b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc2816fdac31480e83d01d6733136944'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:30.682840 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8e38be73-17a9-4807-95a0-ba1a0488c06b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc2816fdac31480e83d01d6733136944'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:30.690831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8e38be73-17a9-4807-95a0-ba1a0488c06b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc2816fdac31480e83d01d6733136944'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:30.692813 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:30.705553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8e38be73-17a9-4807-95a0-ba1a0488c06b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc2816fdac31480e83d01d6733136944'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:30.712535 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8e38be73-17a9-4807-95a0-ba1a0488c06b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc2816fdac31480e83d01d6733136944'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:30.719472 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1310/2597] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:48:30 2022] DELETE /identity/v3/users/cc2816fdac31480e83d01d6733136944 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:48:30.721271 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46983636-ae4f-41f8-8ddf-36571f213854 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:30.723551 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ed85a-05b7-4d0b-ab02-4705e9940797 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:30.723843 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ed85a-05b7-4d0b-ab02-4705e9940797 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:30.724125 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ed85a-05b7-4d0b-ab02-4705e9940797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:30.744906 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-fd7ed85a-05b7-4d0b-ab02-4705e9940797 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:30.746155 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-fd7ed85a-05b7-4d0b-ab02-4705e9940797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:30.774670 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d1ad0-0d27-4d29-a7d0-6abcfb923859 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:30.775916 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d1ad0-0d27-4d29-a7d0-6abcfb923859 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:30.776208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d1ad0-0d27-4d29-a7d0-6abcfb923859 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:30.776486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d1ad0-0d27-4d29-a7d0-6abcfb923859 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:30.778777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8a2d1ad0-0d27-4d29-a7d0-6abcfb923859 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:30.798217 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:30.806997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1311/2598] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:48:30.817585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ed85a-05b7-4d0b-ab02-4705e9940797 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:30.818959 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-fd7ed85a-05b7-4d0b-ab02-4705e9940797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:30.844252 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d1ad0-0d27-4d29-a7d0-6abcfb923859 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:30.845736 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d1ad0-0d27-4d29-a7d0-6abcfb923859 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:30.855857 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1288/2599] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:48:30.877469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ed85a-05b7-4d0b-ab02-4705e9940797 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:30.881275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6d96d-c213-41d9-880e-f9eab1aef77f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:30.881888 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6d96d-c213-41d9-880e-f9eab1aef77f service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:30.882164 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6d96d-c213-41d9-880e-f9eab1aef77f service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:30.882431 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6d96d-c213-41d9-880e-f9eab1aef77f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:30.891732 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6d96d-c213-41d9-880e-f9eab1aef77f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:30.893177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6d96d-c213-41d9-880e-f9eab1aef77f service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:30.903807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1312/2600] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:48:31.202952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d1ad0-0d27-4d29-a7d0-6abcfb923859 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:31.204534 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8a2d1ad0-0d27-4d29-a7d0-6abcfb923859 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:31.225800 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d1ad0-0d27-4d29-a7d0-6abcfb923859 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:31.231436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49c9ca-6395-48a6-a62e-ce525ba865e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:31.232118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49c9ca-6395-48a6-a62e-ce525ba865e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:31.232411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49c9ca-6395-48a6-a62e-ce525ba865e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:31.232758 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49c9ca-6395-48a6-a62e-ce525ba865e0 admin admin] PATH_INFO: `/v3/projects/8b4ccc78886e49bab441df974a59f153` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:31.233854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49c9ca-6395-48a6-a62e-ce525ba865e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b4ccc78886e49bab441df974a59f153)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:31.235744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49c9ca-6395-48a6-a62e-ce525ba865e0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:31.286653 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0c49c9ca-6395-48a6-a62e-ce525ba865e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b4ccc78886e49bab441df974a59f153', 'request_id': 'req-0c49c9ca-6395-48a6-a62e-ce525ba865e0'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:31.293009 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0c49c9ca-6395-48a6-a62e-ce525ba865e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b4ccc78886e49bab441df974a59f153', 'request_id': 'req-0c49c9ca-6395-48a6-a62e-ce525ba865e0'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:31.300651 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0c49c9ca-6395-48a6-a62e-ce525ba865e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b4ccc78886e49bab441df974a59f153', 'request_id': 'req-0c49c9ca-6395-48a6-a62e-ce525ba865e0'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:31.311014 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0c49c9ca-6395-48a6-a62e-ce525ba865e0 admin admin] The token cache is being invalidate because project 8b4ccc78886e49bab441df974a59f153 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:48:31.311301 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0c49c9ca-6395-48a6-a62e-ce525ba865e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:31.311918 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0c49c9ca-6395-48a6-a62e-ce525ba865e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:48:31.313456 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1289/2601] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:48:31 2022] DELETE /identity/v3/projects/8b4ccc78886e49bab441df974a59f153 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:31.318296 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc992fe-04f6-489a-a326-712aa176a745 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:31.318789 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc992fe-04f6-489a-a326-712aa176a745 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:31.318894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc992fe-04f6-489a-a326-712aa176a745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:31.339905 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-dbc992fe-04f6-489a-a326-712aa176a745 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:31.341139 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-dbc992fe-04f6-489a-a326-712aa176a745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:31.410717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1313/2602] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:31.420190 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8efe4f-7714-412e-8031-136d178de7d1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:31.420487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8efe4f-7714-412e-8031-136d178de7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:31.420772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8efe4f-7714-412e-8031-136d178de7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:31.440567 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-fd8efe4f-7714-412e-8031-136d178de7d1 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:31.441882 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fd8efe4f-7714-412e-8031-136d178de7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:31.451081 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1290/2603] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:31.455360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc992fe-04f6-489a-a326-712aa176a745 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:31.466507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc992fe-04f6-489a-a326-712aa176a745 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:31.471521 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92525cf-b82a-4b67-993c-e9e795965fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:31.472222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92525cf-b82a-4b67-993c-e9e795965fca admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:31.472533 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92525cf-b82a-4b67-993c-e9e795965fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:31.472825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92525cf-b82a-4b67-993c-e9e795965fca admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:31.473551 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92525cf-b82a-4b67-993c-e9e795965fca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:48:31.473857 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92525cf-b82a-4b67-993c-e9e795965fca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:31.475462 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92525cf-b82a-4b67-993c-e9e795965fca admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:31.489547 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1314/2604] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:48:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:31.494561 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8efe4f-7714-412e-8031-136d178de7d1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:31.503874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8efe4f-7714-412e-8031-136d178de7d1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:31.508697 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbf858-2362-4ae5-ab0c-b7fe605ff649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:31.509350 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbf858-2362-4ae5-ab0c-b7fe605ff649 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:31.510202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbf858-2362-4ae5-ab0c-b7fe605ff649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:31.510478 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbf858-2362-4ae5-ab0c-b7fe605ff649 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:31.511140 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbf858-2362-4ae5-ab0c-b7fe605ff649 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:31.513023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbf858-2362-4ae5-ab0c-b7fe605ff649 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:31.542299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1291/2605] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:48:31 2022] POST /identity/v3/projects => generated 374 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:31.546435 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92525cf-b82a-4b67-993c-e9e795965fca admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:31.553345 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:31.557725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92525cf-b82a-4b67-993c-e9e795965fca admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:31.613074 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e886a88-030a-4dfe-ac62-24160fa11f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:31.613714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e886a88-030a-4dfe-ac62-24160fa11f4a admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:31.613998 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e886a88-030a-4dfe-ac62-24160fa11f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:31.614272 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e886a88-030a-4dfe-ac62-24160fa11f4a admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:31.614936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e886a88-030a-4dfe-ac62-24160fa11f4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:31.616874 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e886a88-030a-4dfe-ac62-24160fa11f4a admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:31.639183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1315/2606] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:48:31 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:31.643437 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbf858-2362-4ae5-ab0c-b7fe605ff649 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:31.654082 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbf858-2362-4ae5-ab0c-b7fe605ff649 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:31.658849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981950e-69c7-4a49-af13-ed476cd99f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:31.659463 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981950e-69c7-4a49-af13-ed476cd99f07 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:31.659744 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981950e-69c7-4a49-af13-ed476cd99f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:31.660015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981950e-69c7-4a49-af13-ed476cd99f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:31.660730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981950e-69c7-4a49-af13-ed476cd99f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:31.662343 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981950e-69c7-4a49-af13-ed476cd99f07 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:31.672340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1292/2607] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:31 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:31.676290 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e886a88-030a-4dfe-ac62-24160fa11f4a admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:31.686837 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e886a88-030a-4dfe-ac62-24160fa11f4a admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:31.691457 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2606c4f-937e-42fa-883c-1a092f93b87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:31.692067 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2606c4f-937e-42fa-883c-1a092f93b87f admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:31.692356 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2606c4f-937e-42fa-883c-1a092f93b87f admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:31.692635 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2606c4f-937e-42fa-883c-1a092f93b87f admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:31.693358 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2606c4f-937e-42fa-883c-1a092f93b87f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:31.694949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2606c4f-937e-42fa-883c-1a092f93b87f admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:31.706605 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1316/2608] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:48:31 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:48:31.710847 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981950e-69c7-4a49-af13-ed476cd99f07 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:31.721577 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981950e-69c7-4a49-af13-ed476cd99f07 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:31.726232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3132a2-5ede-40c2-ac1a-d8d7728b0ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:31.726885 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3132a2-5ede-40c2-ac1a-d8d7728b0ca7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:31.727169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3132a2-5ede-40c2-ac1a-d8d7728b0ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:31.727444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3132a2-5ede-40c2-ac1a-d8d7728b0ca7 admin admin] PATH_INFO: `/v3/projects/90a51464bbe04f8eb92cf3fe58374256/users/96b0af2a0a8440a98e8092594f090bea/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:31.743623 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3132a2-5ede-40c2-ac1a-d8d7728b0ca7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a51464bbe04f8eb92cf3fe58374256, user_id=96b0af2a0a8440a98e8092594f090bea, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:31.747501 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3132a2-5ede-40c2-ac1a-d8d7728b0ca7 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:31.761458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1293/2609] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:48:31 2022] PUT /identity/v3/projects/90a51464bbe04f8eb92cf3fe58374256/users/96b0af2a0a8440a98e8092594f090bea/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:31.766642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d8c95-50bf-42c8-928c-c44fba54801f None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:31.766935 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d8c95-50bf-42c8-928c-c44fba54801f None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:31.767225 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d8c95-50bf-42c8-928c-c44fba54801f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:31.786443 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-7c9d8c95-50bf-42c8-928c-c44fba54801f None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:31.787656 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-7c9d8c95-50bf-42c8-928c-c44fba54801f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:31.847207 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1317/2610] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:31.856816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3132a2-5ede-40c2-ac1a-d8d7728b0ca7 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:31.858674 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-df3132a2-5ede-40c2-ac1a-d8d7728b0ca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:31.902230 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:31.919932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3132a2-5ede-40c2-ac1a-d8d7728b0ca7 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:31.923742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada67b95-4c30-4fdd-86b5-ce05886b7fe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:31.924424 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada67b95-4c30-4fdd-86b5-ce05886b7fe8 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:31.924710 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada67b95-4c30-4fdd-86b5-ce05886b7fe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:31.924986 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada67b95-4c30-4fdd-86b5-ce05886b7fe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:31.935445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada67b95-4c30-4fdd-86b5-ce05886b7fe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:31.936870 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada67b95-4c30-4fdd-86b5-ce05886b7fe8 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:31.946436 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1294/2611] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:31 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:48:32.631465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e826c3d-8e00-49b8-9510-d421b9210a26 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:32.631972 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e826c3d-8e00-49b8-9510-d421b9210a26 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:32.632242 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e826c3d-8e00-49b8-9510-d421b9210a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:32.665098 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-4e826c3d-8e00-49b8-9510-d421b9210a26 None None] MFA Rules not processed for user `96b0af2a0a8440a98e8092594f090bea`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:32.667217 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-4e826c3d-8e00-49b8-9510-d421b9210a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:32.748222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1318/2612] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:32.762364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada67b95-4c30-4fdd-86b5-ce05886b7fe8 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:32.763951 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ada67b95-4c30-4fdd-86b5-ce05886b7fe8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:32.850794 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada67b95-4c30-4fdd-86b5-ce05886b7fe8 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:32.855082 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfb677-c0b8-4c0e-8a09-dcae20fac5cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:32.855715 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfb677-c0b8-4c0e-8a09-dcae20fac5cc service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:32.855998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfb677-c0b8-4c0e-8a09-dcae20fac5cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:32.856272 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfb677-c0b8-4c0e-8a09-dcae20fac5cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:32.864923 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dfb677-c0b8-4c0e-8a09-dcae20fac5cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:32.866385 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dfb677-c0b8-4c0e-8a09-dcae20fac5cc service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:32.875821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1295/2613] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:48:34.241591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90edd8-e5b9-42ff-a5a4-fb603f077ae3 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:34.242075 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90edd8-e5b9-42ff-a5a4-fb603f077ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:34.242194 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90edd8-e5b9-42ff-a5a4-fb603f077ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:34.263336 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-3b90edd8-e5b9-42ff-a5a4-fb603f077ae3 None None] MFA Rules not processed for user `96b0af2a0a8440a98e8092594f090bea`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:34.264543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3b90edd8-e5b9-42ff-a5a4-fb603f077ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:34.271784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1319/2614] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:34.313830 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfb677-c0b8-4c0e-8a09-dcae20fac5cc service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:34.315209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-85dfb677-c0b8-4c0e-8a09-dcae20fac5cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:34.372235 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfb677-c0b8-4c0e-8a09-dcae20fac5cc service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:34.375885 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f39ec5-94a3-489f-8842-586417b64808 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd959ddd087724547a11c3d4c796d8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:34.376503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f39ec5-94a3-489f-8842-586417b64808 service glance] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:34.376788 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f39ec5-94a3-489f-8842-586417b64808 service glance] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:34.377066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f39ec5-94a3-489f-8842-586417b64808 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:34.378839 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c9f39ec5-94a3-489f-8842-586417b64808 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:34.463662 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f39ec5-94a3-489f-8842-586417b64808 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:34.465028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f39ec5-94a3-489f-8842-586417b64808 service glance] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:34.475916 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1296/2615] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:34 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:48:35.029353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90edd8-e5b9-42ff-a5a4-fb603f077ae3 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:35.030864 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3b90edd8-e5b9-42ff-a5a4-fb603f077ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:35.044820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90edd8-e5b9-42ff-a5a4-fb603f077ae3 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:35.048518 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94cbb72-2d8b-4ceb-90c2-a248b6062499 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:35.049244 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cbb72-2d8b-4ceb-90c2-a248b6062499 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:35.049595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cbb72-2d8b-4ceb-90c2-a248b6062499 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:35.049947 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cbb72-2d8b-4ceb-90c2-a248b6062499 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:35.052219 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b94cbb72-2d8b-4ceb-90c2-a248b6062499 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:35.065159 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:35.070049 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94cbb72-2d8b-4ceb-90c2-a248b6062499 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:35.071059 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94cbb72-2d8b-4ceb-90c2-a248b6062499 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:35.088576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1320/2616] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:35 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:48:44.471063 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=94 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:45.083460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:53.418546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a24d67-992d-4d56-95a4-0ec21b7a1c91 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:53.418546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a24d67-992d-4d56-95a4-0ec21b7a1c91 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:53.418546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a24d67-992d-4d56-95a4-0ec21b7a1c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:53.432639 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d9a24d67-992d-4d56-95a4-0ec21b7a1c91 None None] MFA Rules not processed for user `96b0af2a0a8440a98e8092594f090bea`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:53.433883 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d9a24d67-992d-4d56-95a4-0ec21b7a1c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:53.440771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1297/2617] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:48:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:48:54.820710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94cbb72-2d8b-4ceb-90c2-a248b6062499 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:54.823366 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b94cbb72-2d8b-4ceb-90c2-a248b6062499 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:54.835170 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94cbb72-2d8b-4ceb-90c2-a248b6062499 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:54.839779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1706f-a38f-4d41-8957-ebb3966e455c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:54.840565 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1706f-a38f-4d41-8957-ebb3966e455c service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:54.840941 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1706f-a38f-4d41-8957-ebb3966e455c service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:54.841488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1706f-a38f-4d41-8957-ebb3966e455c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:54.843520 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c2e1706f-a38f-4d41-8957-ebb3966e455c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:54.939040 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e1706f-a38f-4d41-8957-ebb3966e455c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:54.940262 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e1706f-a38f-4d41-8957-ebb3966e455c service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:54.961254 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1321/2618] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:48:54 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:49:03.426253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:04.958545 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=41 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:06.252649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a24d67-992d-4d56-95a4-0ec21b7a1c91 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:06.254170 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d9a24d67-992d-4d56-95a4-0ec21b7a1c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:06.268562 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a24d67-992d-4d56-95a4-0ec21b7a1c91 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:06.273086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f14e3-c904-4d1d-9d33-beb8b439a6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:06.273856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f14e3-c904-4d1d-9d33-beb8b439a6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:06.274141 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f14e3-c904-4d1d-9d33-beb8b439a6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:06.274421 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f14e3-c904-4d1d-9d33-beb8b439a6ee admin admin] PATH_INFO: `/v3/users/28171410d16f4a65854540d373ebb3fb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:06.275573 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f14e3-c904-4d1d-9d33-beb8b439a6ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=28171410d16f4a65854540d373ebb3fb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:06.277721 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f14e3-c904-4d1d-9d33-beb8b439a6ee admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:06.324695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bf9f14e3-c904-4d1d-9d33-beb8b439a6ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28171410d16f4a65854540d373ebb3fb'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:06.331789 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bf9f14e3-c904-4d1d-9d33-beb8b439a6ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28171410d16f4a65854540d373ebb3fb'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:06.332503 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bf9f14e3-c904-4d1d-9d33-beb8b439a6ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28171410d16f4a65854540d373ebb3fb'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:06.341325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bf9f14e3-c904-4d1d-9d33-beb8b439a6ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28171410d16f4a65854540d373ebb3fb'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:06.342038 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bf9f14e3-c904-4d1d-9d33-beb8b439a6ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28171410d16f4a65854540d373ebb3fb'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:06.361863 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1298/2619] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:49:06 2022] DELETE /identity/v3/users/28171410d16f4a65854540d373ebb3fb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:49:06.366442 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126709d-f26b-426b-bd7e-860297e7676d None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:06.366886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126709d-f26b-426b-bd7e-860297e7676d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:06.366997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126709d-f26b-426b-bd7e-860297e7676d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:06.387404 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-6126709d-f26b-426b-bd7e-860297e7676d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:49:06.388601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-6126709d-f26b-426b-bd7e-860297e7676d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:06.451725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1322/2620] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:49:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:49:06.458574 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f14e3-c904-4d1d-9d33-beb8b439a6ee admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:06.460021 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bf9f14e3-c904-4d1d-9d33-beb8b439a6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:06.536194 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f14e3-c904-4d1d-9d33-beb8b439a6ee admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:06.539896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd35530-af0c-4dd1-ac4b-0edfd91ee7ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:06.540519 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd35530-af0c-4dd1-ac4b-0edfd91ee7ab service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:06.540803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd35530-af0c-4dd1-ac4b-0edfd91ee7ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:06.541077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd35530-af0c-4dd1-ac4b-0edfd91ee7ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:06.551964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd35530-af0c-4dd1-ac4b-0edfd91ee7ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:06.555035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd35530-af0c-4dd1-ac4b-0edfd91ee7ab service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:06.567993 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1299/2621] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:49:06 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:49:06.860675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126709d-f26b-426b-bd7e-860297e7676d None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:06.862172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-6126709d-f26b-426b-bd7e-860297e7676d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:06.874439 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126709d-f26b-426b-bd7e-860297e7676d None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:06.879699 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc8901-4675-4207-9a91-7cf3f378dffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:06.880639 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc8901-4675-4207-9a91-7cf3f378dffb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:06.880923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc8901-4675-4207-9a91-7cf3f378dffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:06.881232 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc8901-4675-4207-9a91-7cf3f378dffb admin admin] PATH_INFO: `/v3/projects/144fce8b4197412e969dae5a4c545730` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:06.882403 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc8901-4675-4207-9a91-7cf3f378dffb admin admin] RBAC: Authorizing `identity:delete_project(project_id=144fce8b4197412e969dae5a4c545730)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:06.884316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc8901-4675-4207-9a91-7cf3f378dffb admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:06.934506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-84cc8901-4675-4207-9a91-7cf3f378dffb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '144fce8b4197412e969dae5a4c545730', 'request_id': 'req-84cc8901-4675-4207-9a91-7cf3f378dffb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:06.940686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-84cc8901-4675-4207-9a91-7cf3f378dffb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '144fce8b4197412e969dae5a4c545730', 'request_id': 'req-84cc8901-4675-4207-9a91-7cf3f378dffb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:06.948773 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-84cc8901-4675-4207-9a91-7cf3f378dffb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '144fce8b4197412e969dae5a4c545730', 'request_id': 'req-84cc8901-4675-4207-9a91-7cf3f378dffb'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:06.959627 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-84cc8901-4675-4207-9a91-7cf3f378dffb admin admin] The token cache is being invalidate because project 144fce8b4197412e969dae5a4c545730 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:49:06.959917 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-84cc8901-4675-4207-9a91-7cf3f378dffb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:06.960538 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-84cc8901-4675-4207-9a91-7cf3f378dffb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:06.962102 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1323/2622] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:49:06 2022] DELETE /identity/v3/projects/144fce8b4197412e969dae5a4c545730 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:49:12.503576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f518fb-a9c9-4056-a864-1441fb2a4741 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:12.504068 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f518fb-a9c9-4056-a864-1441fb2a4741 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:12.504158 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f518fb-a9c9-4056-a864-1441fb2a4741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:12.523982 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a8f518fb-a9c9-4056-a864-1441fb2a4741 None None] MFA Rules not processed for user `1cd36b543bd9487ebd8ce578e9a76c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:49:12.525206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a8f518fb-a9c9-4056-a864-1441fb2a4741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:12.580429 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1300/2623] 10.222.0.100 () {58 vars in 1014 bytes} [Tue Aug 2 09:49:12 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:49:12.627599 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc8901-4675-4207-9a91-7cf3f378dffb admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:12.629256 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-84cc8901-4675-4207-9a91-7cf3f378dffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:12.700662 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc8901-4675-4207-9a91-7cf3f378dffb admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:12.708604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a851b36-dcd7-4a37-bddc-9eb8976657cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50cb532b58ed4951ba960a0a7d579b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:12.709654 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a851b36-dcd7-4a37-bddc-9eb8976657cb service swift] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:12.709948 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a851b36-dcd7-4a37-bddc-9eb8976657cb service swift] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:12.710232 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a851b36-dcd7-4a37-bddc-9eb8976657cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:12.720575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a851b36-dcd7-4a37-bddc-9eb8976657cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:12.722197 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a851b36-dcd7-4a37-bddc-9eb8976657cb service swift] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:12.730291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1324/2624] 10.222.0.100 () {60 vars in 1389 bytes} [Tue Aug 2 09:49:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:49:22.573204 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=80,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:22.727231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:27.916107 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffd230-e598-430b-903f-7c7df16c8690 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:27.916564 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffd230-e598-430b-903f-7c7df16c8690 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:27.916671 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffd230-e598-430b-903f-7c7df16c8690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:27.937941 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3fffd230-e598-430b-903f-7c7df16c8690 None None] MFA Rules not processed for user `1cd36b543bd9487ebd8ce578e9a76c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:49:27.939151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3fffd230-e598-430b-903f-7c7df16c8690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:27.946470 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1301/2625] 10.222.0.100 () {58 vars in 972 bytes} [Tue Aug 2 09:49:27 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:49:27.960498 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a851b36-dcd7-4a37-bddc-9eb8976657cb service swift] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:27.972115 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a851b36-dcd7-4a37-bddc-9eb8976657cb service swift] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:27.978433 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651ac6a-a136-4d58-aaf1-6b75a42be4e3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50cb532b58ed4951ba960a0a7d579b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:27.979063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651ac6a-a136-4d58-aaf1-6b75a42be4e3 service swift] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:27.979331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651ac6a-a136-4d58-aaf1-6b75a42be4e3 service swift] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:27.979688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651ac6a-a136-4d58-aaf1-6b75a42be4e3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:27.988958 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651ac6a-a136-4d58-aaf1-6b75a42be4e3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:27.990576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651ac6a-a136-4d58-aaf1-6b75a42be4e3 service swift] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:27.999165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1325/2626] 10.222.0.100 () {60 vars in 1389 bytes} [Tue Aug 2 09:49:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:49:29.870384 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffd230-e598-430b-903f-7c7df16c8690 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:29.871820 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3fffd230-e598-430b-903f-7c7df16c8690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:29.955544 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651ac6a-a136-4d58-aaf1-6b75a42be4e3 service swift] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:29.957014 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-0651ac6a-a136-4d58-aaf1-6b75a42be4e3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:30.044323 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651ac6a-a136-4d58-aaf1-6b75a42be4e3 service swift] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:30.049209 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a89ed8-97e4-4662-ab1b-4d9460cb399c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:30.049860 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a89ed8-97e4-4662-ab1b-4d9460cb399c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:30.050148 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a89ed8-97e4-4662-ab1b-4d9460cb399c admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:30.050427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a89ed8-97e4-4662-ab1b-4d9460cb399c admin admin] PATH_INFO: `/v3/users/96b0af2a0a8440a98e8092594f090bea` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:30.051612 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a89ed8-97e4-4662-ab1b-4d9460cb399c admin admin] RBAC: Authorizing `identity:delete_user(user_id=96b0af2a0a8440a98e8092594f090bea)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:30.053628 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a89ed8-97e4-4662-ab1b-4d9460cb399c admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:30.095076 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffd230-e598-430b-903f-7c7df16c8690 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:30.100345 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bbae4-d98b-4654-abf6-201a8b4f4594 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b40981fe179a43dc81553d3594dad649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:30.101088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bbae4-d98b-4654-abf6-201a8b4f4594 service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:30.101397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bbae4-d98b-4654-abf6-201a8b4f4594 service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:30.101667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bbae4-d98b-4654-abf6-201a8b4f4594 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:30.103565 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d32bbae4-d98b-4654-abf6-201a8b4f4594 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:30.105738 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-34a89ed8-97e4-4662-ab1b-4d9460cb399c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96b0af2a0a8440a98e8092594f090bea'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:30.106414 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-34a89ed8-97e4-4662-ab1b-4d9460cb399c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96b0af2a0a8440a98e8092594f090bea'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:30.114845 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-34a89ed8-97e4-4662-ab1b-4d9460cb399c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96b0af2a0a8440a98e8092594f090bea'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:30.117668 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32bbae4-d98b-4654-abf6-201a8b4f4594 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:30.119313 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32bbae4-d98b-4654-abf6-201a8b4f4594 service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:30.130870 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-34a89ed8-97e4-4662-ab1b-4d9460cb399c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96b0af2a0a8440a98e8092594f090bea'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:30.131617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1302/2627] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:49:29 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:49:30.138352 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-34a89ed8-97e4-4662-ab1b-4d9460cb399c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96b0af2a0a8440a98e8092594f090bea'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:30.145514 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1326/2628] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:49:29 2022] DELETE /identity/v3/users/96b0af2a0a8440a98e8092594f090bea => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:49:30.323058 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bbae4-d98b-4654-abf6-201a8b4f4594 service placement] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:30.335541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bbae4-d98b-4654-abf6-201a8b4f4594 service placement] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:30.407486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79ac13-9429-4e66-ab45-23fe52d225fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:30.408135 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79ac13-9429-4e66-ab45-23fe52d225fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:30.408415 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79ac13-9429-4e66-ab45-23fe52d225fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:30.408688 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79ac13-9429-4e66-ab45-23fe52d225fb admin admin] PATH_INFO: `/v3/projects/90a51464bbe04f8eb92cf3fe58374256` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:30.409874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79ac13-9429-4e66-ab45-23fe52d225fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=90a51464bbe04f8eb92cf3fe58374256)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:30.411763 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79ac13-9429-4e66-ab45-23fe52d225fb admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:30.461431 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dd79ac13-9429-4e66-ab45-23fe52d225fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90a51464bbe04f8eb92cf3fe58374256', 'request_id': 'req-dd79ac13-9429-4e66-ab45-23fe52d225fb'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:30.469932 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dd79ac13-9429-4e66-ab45-23fe52d225fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90a51464bbe04f8eb92cf3fe58374256', 'request_id': 'req-dd79ac13-9429-4e66-ab45-23fe52d225fb'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:30.478767 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dd79ac13-9429-4e66-ab45-23fe52d225fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90a51464bbe04f8eb92cf3fe58374256', 'request_id': 'req-dd79ac13-9429-4e66-ab45-23fe52d225fb'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:30.489877 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dd79ac13-9429-4e66-ab45-23fe52d225fb admin admin] The token cache is being invalidate because project 90a51464bbe04f8eb92cf3fe58374256 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:49:30.490298 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dd79ac13-9429-4e66-ab45-23fe52d225fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:30.490936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dd79ac13-9429-4e66-ab45-23fe52d225fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:30.492541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1303/2629] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:49:30 2022] DELETE /identity/v3/projects/90a51464bbe04f8eb92cf3fe58374256 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:49:30.497614 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c237c4-d9fb-4eea-a67a-69d2de6510d3 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:30.497844 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c237c4-d9fb-4eea-a67a-69d2de6510d3 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:30.498153 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c237c4-d9fb-4eea-a67a-69d2de6510d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:30.519543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-77c237c4-d9fb-4eea-a67a-69d2de6510d3 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:49:30.520763 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-77c237c4-d9fb-4eea-a67a-69d2de6510d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:30.582281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1327/2630] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:49:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:49:30.590038 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45d286-1243-44a5-900b-ff69fc3bb123 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:30.590330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45d286-1243-44a5-900b-ff69fc3bb123 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:30.590610 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45d286-1243-44a5-900b-ff69fc3bb123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:30.611023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1f45d286-1243-44a5-900b-ff69fc3bb123 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:49:30.612224 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1f45d286-1243-44a5-900b-ff69fc3bb123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:30.620222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1304/2631] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:49:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:49:30.625644 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c237c4-d9fb-4eea-a67a-69d2de6510d3 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:30.635709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c237c4-d9fb-4eea-a67a-69d2de6510d3 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:30.641688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dbe3c-141b-4fdf-ae12-1d1bb6774640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:30.642325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dbe3c-141b-4fdf-ae12-1d1bb6774640 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:30.642605 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dbe3c-141b-4fdf-ae12-1d1bb6774640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:30.642879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dbe3c-141b-4fdf-ae12-1d1bb6774640 admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:30.643563 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165dbe3c-141b-4fdf-ae12-1d1bb6774640 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:30.643866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165dbe3c-141b-4fdf-ae12-1d1bb6774640 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:30.645501 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165dbe3c-141b-4fdf-ae12-1d1bb6774640 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:30.647057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:30.659967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1328/2632] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:49:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:30.664411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45d286-1243-44a5-900b-ff69fc3bb123 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:30.674061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45d286-1243-44a5-900b-ff69fc3bb123 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:30.678683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85e3af-7edb-453a-aba7-e648ee2e024e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:30.679303 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85e3af-7edb-453a-aba7-e648ee2e024e admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:30.679582 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85e3af-7edb-453a-aba7-e648ee2e024e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:30.679880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85e3af-7edb-453a-aba7-e648ee2e024e admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:30.680522 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85e3af-7edb-453a-aba7-e648ee2e024e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:30.682448 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85e3af-7edb-453a-aba7-e648ee2e024e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:30.687800 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:30.713439 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1305/2633] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:49:30 2022] POST /identity/v3/projects => generated 398 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:49:30.718783 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dbe3c-141b-4fdf-ae12-1d1bb6774640 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:30.729963 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dbe3c-141b-4fdf-ae12-1d1bb6774640 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:30.790626 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5903ab-5b67-4d83-bdbf-ecd454a3de7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:30.791215 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5903ab-5b67-4d83-bdbf-ecd454a3de7d admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:30.791502 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5903ab-5b67-4d83-bdbf-ecd454a3de7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:30.791780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5903ab-5b67-4d83-bdbf-ecd454a3de7d admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:30.792469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5903ab-5b67-4d83-bdbf-ecd454a3de7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:30.794439 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5903ab-5b67-4d83-bdbf-ecd454a3de7d admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:30.819405 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1329/2634] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:49:30 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:49:30.824071 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85e3af-7edb-453a-aba7-e648ee2e024e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:30.834497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85e3af-7edb-453a-aba7-e648ee2e024e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:30.839275 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde5afa-4cc4-4d96-ac82-d7ed4a4013d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:30.839894 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde5afa-4cc4-4d96-ac82-d7ed4a4013d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:30.840180 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde5afa-4cc4-4d96-ac82-d7ed4a4013d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:30.840461 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde5afa-4cc4-4d96-ac82-d7ed4a4013d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:30.841197 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde5afa-4cc4-4d96-ac82-d7ed4a4013d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:30.843505 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde5afa-4cc4-4d96-ac82-d7ed4a4013d0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:30.854262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1306/2635] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:49:30 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:49:30.868042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5903ab-5b67-4d83-bdbf-ecd454a3de7d admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:30.873926 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5903ab-5b67-4d83-bdbf-ecd454a3de7d admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:30.881566 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2cc6bd-46e9-4c07-8655-f21b53723e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:30.882193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cc6bd-46e9-4c07-8655-f21b53723e68 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:30.882472 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cc6bd-46e9-4c07-8655-f21b53723e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:30.882747 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cc6bd-46e9-4c07-8655-f21b53723e68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:30.883498 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2cc6bd-46e9-4c07-8655-f21b53723e68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:30.885137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2cc6bd-46e9-4c07-8655-f21b53723e68 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:30.896601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1330/2636] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:49:30 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:49:30.901042 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde5afa-4cc4-4d96-ac82-d7ed4a4013d0 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:30.912348 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde5afa-4cc4-4d96-ac82-d7ed4a4013d0 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:30.917194 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbab838-ff34-4965-b9a7-60fcdbdb9604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:30.917892 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbab838-ff34-4965-b9a7-60fcdbdb9604 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:30.918171 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbab838-ff34-4965-b9a7-60fcdbdb9604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:30.918445 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbab838-ff34-4965-b9a7-60fcdbdb9604 admin admin] PATH_INFO: `/v3/projects/62d498b1f85a454994f0646fa4182f6e/users/80fe242b13ee4ca483a51fea38a09ac0/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:30.935216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbab838-ff34-4965-b9a7-60fcdbdb9604 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62d498b1f85a454994f0646fa4182f6e, user_id=80fe242b13ee4ca483a51fea38a09ac0, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:30.938251 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbab838-ff34-4965-b9a7-60fcdbdb9604 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:30.952674 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1307/2637] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:49:30 2022] PUT /identity/v3/projects/62d498b1f85a454994f0646fa4182f6e/users/80fe242b13ee4ca483a51fea38a09ac0/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:49:30.964287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c4bf0-9b3f-401c-aebf-00d9289ba8bd None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:30.964787 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c4bf0-9b3f-401c-aebf-00d9289ba8bd None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:30.964910 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c4bf0-9b3f-401c-aebf-00d9289ba8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:30.997824 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-e69c4bf0-9b3f-401c-aebf-00d9289ba8bd None None] MFA Rules not processed for user `80fe242b13ee4ca483a51fea38a09ac0`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:49:30.999222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e69c4bf0-9b3f-401c-aebf-00d9289ba8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:31.079075 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1331/2638] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:49:30 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:49:31.090521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbab838-ff34-4965-b9a7-60fcdbdb9604 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:31.093713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1cbab838-ff34-4965-b9a7-60fcdbdb9604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:31.150364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbab838-ff34-4965-b9a7-60fcdbdb9604 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:31.154233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb0bbe-3adf-47c2-8344-d75e9724e326 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:31.155449 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb0bbe-3adf-47c2-8344-d75e9724e326 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:31.155735 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb0bbe-3adf-47c2-8344-d75e9724e326 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:31.156013 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb0bbe-3adf-47c2-8344-d75e9724e326 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:31.164568 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb0bbe-3adf-47c2-8344-d75e9724e326 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:31.166090 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb0bbe-3adf-47c2-8344-d75e9724e326 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:31.175045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1308/2639] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:49:31 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:49:39.303229 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c4bf0-9b3f-401c-aebf-00d9289ba8bd None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:39.377700 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c4bf0-9b3f-401c-aebf-00d9289ba8bd None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:39.382613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20625c8d-1087-401d-ab8f-14608cfb0aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:39.383246 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20625c8d-1087-401d-ab8f-14608cfb0aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:39.383522 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20625c8d-1087-401d-ab8f-14608cfb0aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:39.383791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20625c8d-1087-401d-ab8f-14608cfb0aec admin admin] PATH_INFO: `/v3/users/80fe242b13ee4ca483a51fea38a09ac0` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:39.384955 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20625c8d-1087-401d-ab8f-14608cfb0aec admin admin] RBAC: Authorizing `identity:delete_user(user_id=80fe242b13ee4ca483a51fea38a09ac0)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:39.386895 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20625c8d-1087-401d-ab8f-14608cfb0aec admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:39.389625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:39.436336 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-20625c8d-1087-401d-ab8f-14608cfb0aec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80fe242b13ee4ca483a51fea38a09ac0'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:39.437140 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-20625c8d-1087-401d-ab8f-14608cfb0aec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80fe242b13ee4ca483a51fea38a09ac0'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:39.445376 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-20625c8d-1087-401d-ab8f-14608cfb0aec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80fe242b13ee4ca483a51fea38a09ac0'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:39.458623 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-20625c8d-1087-401d-ab8f-14608cfb0aec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80fe242b13ee4ca483a51fea38a09ac0'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:39.466235 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-20625c8d-1087-401d-ab8f-14608cfb0aec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80fe242b13ee4ca483a51fea38a09ac0'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:39.473061 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1332/2640] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:49:39 2022] DELETE /identity/v3/users/80fe242b13ee4ca483a51fea38a09ac0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:49:39.477781 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76d48c-a0eb-4bf0-b507-e039e8e198de None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:39.478238 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76d48c-a0eb-4bf0-b507-e039e8e198de None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:39.478456 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76d48c-a0eb-4bf0-b507-e039e8e198de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:39.501717 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9d76d48c-a0eb-4bf0-b507-e039e8e198de None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:49:39.503513 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9d76d48c-a0eb-4bf0-b507-e039e8e198de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:39.572537 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1309/2641] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:49:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:49:39.582606 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20625c8d-1087-401d-ab8f-14608cfb0aec admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:39.584033 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-20625c8d-1087-401d-ab8f-14608cfb0aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:39.643536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20625c8d-1087-401d-ab8f-14608cfb0aec admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:39.647157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0cb5d5-9409-486e-93dd-b1fc4b29b0f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:39.647774 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cb5d5-9409-486e-93dd-b1fc4b29b0f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:39.648054 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cb5d5-9409-486e-93dd-b1fc4b29b0f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:39.648331 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cb5d5-9409-486e-93dd-b1fc4b29b0f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:39.657779 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0cb5d5-9409-486e-93dd-b1fc4b29b0f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:39.659222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0cb5d5-9409-486e-93dd-b1fc4b29b0f8 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:39.669458 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1333/2642] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:49:39 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:49:39.948518 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76d48c-a0eb-4bf0-b507-e039e8e198de None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:39.950077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9d76d48c-a0eb-4bf0-b507-e039e8e198de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:39.963236 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76d48c-a0eb-4bf0-b507-e039e8e198de None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:39.969253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1e0a4-e20e-4f62-a296-01a12792a5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:39.970038 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1e0a4-e20e-4f62-a296-01a12792a5fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:39.970500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1e0a4-e20e-4f62-a296-01a12792a5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:39.970968 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1e0a4-e20e-4f62-a296-01a12792a5fd admin admin] PATH_INFO: `/v3/projects/62d498b1f85a454994f0646fa4182f6e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:39.972985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1e0a4-e20e-4f62-a296-01a12792a5fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=62d498b1f85a454994f0646fa4182f6e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:39.975943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1e0a4-e20e-4f62-a296-01a12792a5fd admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:40.027513 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:40.029010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f5b1e0a4-e20e-4f62-a296-01a12792a5fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62d498b1f85a454994f0646fa4182f6e', 'request_id': 'req-f5b1e0a4-e20e-4f62-a296-01a12792a5fd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:40.036524 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f5b1e0a4-e20e-4f62-a296-01a12792a5fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62d498b1f85a454994f0646fa4182f6e', 'request_id': 'req-f5b1e0a4-e20e-4f62-a296-01a12792a5fd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:40.044054 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f5b1e0a4-e20e-4f62-a296-01a12792a5fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62d498b1f85a454994f0646fa4182f6e', 'request_id': 'req-f5b1e0a4-e20e-4f62-a296-01a12792a5fd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:40.055397 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f5b1e0a4-e20e-4f62-a296-01a12792a5fd admin admin] The token cache is being invalidate because project 62d498b1f85a454994f0646fa4182f6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:49:40.055688 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f5b1e0a4-e20e-4f62-a296-01a12792a5fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:40.056330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f5b1e0a4-e20e-4f62-a296-01a12792a5fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:49:40.057876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1310/2643] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:49:39 2022] DELETE /identity/v3/projects/62d498b1f85a454994f0646fa4182f6e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:49:49.664578 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:50.052276 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=7,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:50:41.135334 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0cb5d5-9409-486e-93dd-b1fc4b29b0f8 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:50:41.137381 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ad0cb5d5-9409-486e-93dd-b1fc4b29b0f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:50:41.249355 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0cb5d5-9409-486e-93dd-b1fc4b29b0f8 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:50:41.315949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9834f-6c14-4392-8cd1-fa0cc936194a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b40981fe179a43dc81553d3594dad649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:50:41.316576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9834f-6c14-4392-8cd1-fa0cc936194a service placement] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:50:41.316866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9834f-6c14-4392-8cd1-fa0cc936194a service placement] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:50:41.317171 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9834f-6c14-4392-8cd1-fa0cc936194a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:50:41.318948 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-6cc9834f-6c14-4392-8cd1-fa0cc936194a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:50:41.330585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9834f-6c14-4392-8cd1-fa0cc936194a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:50:41.332172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9834f-6c14-4392-8cd1-fa0cc936194a service placement] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:50:41.344111 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1334/2644] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:50:41 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:50:51.337856 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=75 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:54.525492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1e0a4-e20e-4f62-a296-01a12792a5fd admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:51:54.527556 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f5b1e0a4-e20e-4f62-a296-01a12792a5fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:54.697732 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1e0a4-e20e-4f62-a296-01a12792a5fd admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:51:54.704305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7077f-a306-49dc-a96d-9531215faafa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:51:54.705128 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7077f-a306-49dc-a96d-9531215faafa service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:54.705431 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7077f-a306-49dc-a96d-9531215faafa service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:54.705712 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7077f-a306-49dc-a96d-9531215faafa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:54.708328 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0fb7077f-a306-49dc-a96d-9531215faafa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:54.723918 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb7077f-a306-49dc-a96d-9531215faafa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:51:54.726037 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb7077f-a306-49dc-a96d-9531215faafa service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:51:54.742711 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1311/2645] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:51:54 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:51:54.818558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9834f-6c14-4392-8cd1-fa0cc936194a service placement] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:51:54.821648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-6cc9834f-6c14-4392-8cd1-fa0cc936194a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:54.943037 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9834f-6c14-4392-8cd1-fa0cc936194a service placement] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:51:54.960766 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2902ab-1a1d-4c2c-b8ab-1775c6c460a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:51:54.962503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2902ab-1a1d-4c2c-b8ab-1775c6c460a0 service nova] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:54.962983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2902ab-1a1d-4c2c-b8ab-1775c6c460a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:54.963272 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2902ab-1a1d-4c2c-b8ab-1775c6c460a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:54.966423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-7b2902ab-1a1d-4c2c-b8ab-1775c6c460a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:54.992102 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2902ab-1a1d-4c2c-b8ab-1775c6c460a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:51:54.996787 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2902ab-1a1d-4c2c-b8ab-1775c6c460a0 service nova] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:51:55.045905 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1335/2646] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:51:54 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:51:56.044295 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7077f-a306-49dc-a96d-9531215faafa service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:51:56.056532 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7077f-a306-49dc-a96d-9531215faafa service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:51:56.064069 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719435d-1b54-4dc7-b132-49523002e764 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:51:56.064742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719435d-1b54-4dc7-b132-49523002e764 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:56.065032 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719435d-1b54-4dc7-b132-49523002e764 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:56.065336 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719435d-1b54-4dc7-b132-49523002e764 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:56.067237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2719435d-1b54-4dc7-b132-49523002e764 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:56.144849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719435d-1b54-4dc7-b132-49523002e764 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:51:56.146911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719435d-1b54-4dc7-b132-49523002e764 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:51:56.161457 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1312/2647] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:51:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:51:59.568646 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85693ec9-732d-4716-b638-fffaa113c3e3 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:59.569439 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85693ec9-732d-4716-b638-fffaa113c3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:59.569439 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85693ec9-732d-4716-b638-fffaa113c3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:59.614446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-85693ec9-732d-4716-b638-fffaa113c3e3 None None] MFA Rules not processed for user `18728d1c3451497480556461b4ec80d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:59.615660 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-85693ec9-732d-4716-b638-fffaa113c3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:59.622701 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1336/2648] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:51:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:51:59.892693 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719435d-1b54-4dc7-b132-49523002e764 service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:51:59.895114 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2719435d-1b54-4dc7-b132-49523002e764 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:59.992171 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85693ec9-732d-4716-b638-fffaa113c3e3 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:51:59.993726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-85693ec9-732d-4716-b638-fffaa113c3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:59.996897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719435d-1b54-4dc7-b132-49523002e764 service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:00.001395 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e168f94-86cc-44a2-b90d-d74bb3933229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:00.002097 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e168f94-86cc-44a2-b90d-d74bb3933229 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:00.002334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e168f94-86cc-44a2-b90d-d74bb3933229 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:00.002597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e168f94-86cc-44a2-b90d-d74bb3933229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:00.005608 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7e168f94-86cc-44a2-b90d-d74bb3933229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:00.008217 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85693ec9-732d-4716-b638-fffaa113c3e3 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:00.011917 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe384a4-a1e1-418b-a673-90b540f68b7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:00.012566 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe384a4-a1e1-418b-a673-90b540f68b7d service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:00.012846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe384a4-a1e1-418b-a673-90b540f68b7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:00.013137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe384a4-a1e1-418b-a673-90b540f68b7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:00.014891 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-bfe384a4-a1e1-418b-a673-90b540f68b7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:00.025846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe384a4-a1e1-418b-a673-90b540f68b7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:00.028253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe384a4-a1e1-418b-a673-90b540f68b7d service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:00.031118 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:00.038647 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1337/2649] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:51:59 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:52:00.093398 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e168f94-86cc-44a2-b90d-d74bb3933229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:00.095056 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e168f94-86cc-44a2-b90d-d74bb3933229 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:00.107903 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1313/2650] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:51:59 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:00.503099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe384a4-a1e1-418b-a673-90b540f68b7d service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:00.505797 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-bfe384a4-a1e1-418b-a673-90b540f68b7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:00.525386 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe384a4-a1e1-418b-a673-90b540f68b7d service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:00.532028 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19686452-3bea-4f38-8435-ff51c81ec783 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b40981fe179a43dc81553d3594dad649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:00.532664 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19686452-3bea-4f38-8435-ff51c81ec783 service placement] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:00.533011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19686452-3bea-4f38-8435-ff51c81ec783 service placement] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:00.533259 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19686452-3bea-4f38-8435-ff51c81ec783 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:00.535242 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-19686452-3bea-4f38-8435-ff51c81ec783 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:00.552843 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19686452-3bea-4f38-8435-ff51c81ec783 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:00.554616 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19686452-3bea-4f38-8435-ff51c81ec783 service placement] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:00.567170 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1338/2651] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:52:00 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:04.617270 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e168f94-86cc-44a2-b90d-d74bb3933229 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:04.619613 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7e168f94-86cc-44a2-b90d-d74bb3933229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:04.639923 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e168f94-86cc-44a2-b90d-d74bb3933229 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:04.645387 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec57aa-0d77-408f-a812-45749063d317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:04.646062 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec57aa-0d77-408f-a812-45749063d317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:04.646347 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec57aa-0d77-408f-a812-45749063d317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:04.646626 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec57aa-0d77-408f-a812-45749063d317 admin admin] PATH_INFO: `/v3/users/6c30f2ab48e74d4690aafe46c4bf682f` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:04.648800 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec57aa-0d77-408f-a812-45749063d317 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c30f2ab48e74d4690aafe46c4bf682f)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:04.651812 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec57aa-0d77-408f-a812-45749063d317 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:04.720320 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c6ec57aa-0d77-408f-a812-45749063d317 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c30f2ab48e74d4690aafe46c4bf682f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:04.727357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c6ec57aa-0d77-408f-a812-45749063d317 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c30f2ab48e74d4690aafe46c4bf682f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:04.728101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c6ec57aa-0d77-408f-a812-45749063d317 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c30f2ab48e74d4690aafe46c4bf682f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:04.736566 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c6ec57aa-0d77-408f-a812-45749063d317 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c30f2ab48e74d4690aafe46c4bf682f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:04.737288 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c6ec57aa-0d77-408f-a812-45749063d317 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c30f2ab48e74d4690aafe46c4bf682f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:04.755590 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1314/2652] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:52:04 2022] DELETE /identity/v3/users/6c30f2ab48e74d4690aafe46c4bf682f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:04.976483 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19686452-3bea-4f38-8435-ff51c81ec783 service placement] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:04.990339 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19686452-3bea-4f38-8435-ff51c81ec783 service placement] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:05.062115 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1997b-a51e-4b2c-9be4-b2a23a0f8c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:05.062753 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1997b-a51e-4b2c-9be4-b2a23a0f8c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:05.063033 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1997b-a51e-4b2c-9be4-b2a23a0f8c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:05.063307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1997b-a51e-4b2c-9be4-b2a23a0f8c30 admin admin] PATH_INFO: `/v3/projects/279f631bf3db4ba7a0aff00226cf5724` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:05.064455 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1997b-a51e-4b2c-9be4-b2a23a0f8c30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=279f631bf3db4ba7a0aff00226cf5724)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:05.066386 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1997b-a51e-4b2c-9be4-b2a23a0f8c30 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:05.127042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-62a1997b-a51e-4b2c-9be4-b2a23a0f8c30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '279f631bf3db4ba7a0aff00226cf5724', 'request_id': 'req-62a1997b-a51e-4b2c-9be4-b2a23a0f8c30'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:05.134403 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-62a1997b-a51e-4b2c-9be4-b2a23a0f8c30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '279f631bf3db4ba7a0aff00226cf5724', 'request_id': 'req-62a1997b-a51e-4b2c-9be4-b2a23a0f8c30'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:05.146285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-62a1997b-a51e-4b2c-9be4-b2a23a0f8c30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '279f631bf3db4ba7a0aff00226cf5724', 'request_id': 'req-62a1997b-a51e-4b2c-9be4-b2a23a0f8c30'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:05.163977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-62a1997b-a51e-4b2c-9be4-b2a23a0f8c30 admin admin] The token cache is being invalidate because project 279f631bf3db4ba7a0aff00226cf5724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:52:05.164462 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-62a1997b-a51e-4b2c-9be4-b2a23a0f8c30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:05.165544 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-62a1997b-a51e-4b2c-9be4-b2a23a0f8c30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:05.169726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1339/2653] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:52:04 2022] DELETE /identity/v3/projects/279f631bf3db4ba7a0aff00226cf5724 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:05.178740 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858beb2-b0f8-4245-92b3-7f550c4b9bc2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:05.179220 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858beb2-b0f8-4245-92b3-7f550c4b9bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:05.179671 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858beb2-b0f8-4245-92b3-7f550c4b9bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:05.201444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f858beb2-b0f8-4245-92b3-7f550c4b9bc2 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:05.202683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f858beb2-b0f8-4245-92b3-7f550c4b9bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:05.303388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1315/2654] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:05.311447 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27acb0e-336a-4f13-a9c8-4e9543f7036d None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:05.311739 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27acb0e-336a-4f13-a9c8-4e9543f7036d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:05.312020 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27acb0e-336a-4f13-a9c8-4e9543f7036d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:05.328238 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:05.334619 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-c27acb0e-336a-4f13-a9c8-4e9543f7036d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:05.336829 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c27acb0e-336a-4f13-a9c8-4e9543f7036d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:05.346798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1340/2655] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:05.351161 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f858beb2-b0f8-4245-92b3-7f550c4b9bc2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:05.373301 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f858beb2-b0f8-4245-92b3-7f550c4b9bc2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:05.378390 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb9461-36c3-4f8a-b8db-2765fd195f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:05.379228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb9461-36c3-4f8a-b8db-2765fd195f3a admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:05.379510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb9461-36c3-4f8a-b8db-2765fd195f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:05.379787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb9461-36c3-4f8a-b8db-2765fd195f3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:05.380490 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb9461-36c3-4f8a-b8db-2765fd195f3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:52:05.380800 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb9461-36c3-4f8a-b8db-2765fd195f3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:05.382464 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb9461-36c3-4f8a-b8db-2765fd195f3a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:05.395893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1316/2656] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:52:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:52:05.400071 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27acb0e-336a-4f13-a9c8-4e9543f7036d None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:05.409771 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27acb0e-336a-4f13-a9c8-4e9543f7036d None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:05.414771 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db811379-1309-40e1-9140-c74f54953aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:05.415391 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db811379-1309-40e1-9140-c74f54953aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:05.415669 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db811379-1309-40e1-9140-c74f54953aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:05.415967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db811379-1309-40e1-9140-c74f54953aa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:05.416648 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db811379-1309-40e1-9140-c74f54953aa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:05.419407 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db811379-1309-40e1-9140-c74f54953aa2 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:05.453729 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1341/2657] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:52:05 2022] POST /identity/v3/projects => generated 368 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:52:05.457484 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb9461-36c3-4f8a-b8db-2765fd195f3a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:05.471024 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb9461-36c3-4f8a-b8db-2765fd195f3a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:05.481419 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:05.539675 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1aec7f-cfe2-4ed9-b99c-dc82c8e1fe22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:05.540298 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1aec7f-cfe2-4ed9-b99c-dc82c8e1fe22 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:05.540579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1aec7f-cfe2-4ed9-b99c-dc82c8e1fe22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:05.540854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1aec7f-cfe2-4ed9-b99c-dc82c8e1fe22 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:05.541564 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1aec7f-cfe2-4ed9-b99c-dc82c8e1fe22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:05.544316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1aec7f-cfe2-4ed9-b99c-dc82c8e1fe22 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:05.572239 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1317/2658] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:52:05 2022] POST /identity/v3/users => generated 285 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:52:05.577431 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db811379-1309-40e1-9140-c74f54953aa2 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:05.589912 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db811379-1309-40e1-9140-c74f54953aa2 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:05.596122 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccd632-100b-42a1-baa6-1d75a9b1d8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:05.596747 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccd632-100b-42a1-baa6-1d75a9b1d8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:05.597027 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccd632-100b-42a1-baa6-1d75a9b1d8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:05.597349 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccd632-100b-42a1-baa6-1d75a9b1d8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:05.598062 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dccd632-100b-42a1-baa6-1d75a9b1d8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:05.600129 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dccd632-100b-42a1-baa6-1d75a9b1d8e6 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:05.612695 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1342/2659] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:05 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:52:05.617538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1aec7f-cfe2-4ed9-b99c-dc82c8e1fe22 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:05.629927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1aec7f-cfe2-4ed9-b99c-dc82c8e1fe22 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:05.634815 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e4e1d-b724-4593-8478-3aa4105ed4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:05.635443 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e4e1d-b724-4593-8478-3aa4105ed4f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:05.636591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e4e1d-b724-4593-8478-3aa4105ed4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:05.637691 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e4e1d-b724-4593-8478-3aa4105ed4f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:05.638450 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e4e1d-b724-4593-8478-3aa4105ed4f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:05.640366 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e4e1d-b724-4593-8478-3aa4105ed4f1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:05.656081 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1318/2660] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:05 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:52:05.666552 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccd632-100b-42a1-baa6-1d75a9b1d8e6 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:05.685018 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccd632-100b-42a1-baa6-1d75a9b1d8e6 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:05.690876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e6fde-2e9e-4aa3-b994-401b11c082aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:05.691878 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e6fde-2e9e-4aa3-b994-401b11c082aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:05.692361 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e6fde-2e9e-4aa3-b994-401b11c082aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:05.693008 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e6fde-2e9e-4aa3-b994-401b11c082aa admin admin] PATH_INFO: `/v3/projects/fcc9f0aa45954a6284ac0f2f8f0e8403/users/41c73c7a9202466fa6db0933c4ba08c3/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:05.713679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e6fde-2e9e-4aa3-b994-401b11c082aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcc9f0aa45954a6284ac0f2f8f0e8403, user_id=41c73c7a9202466fa6db0933c4ba08c3, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:05.716702 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e6fde-2e9e-4aa3-b994-401b11c082aa admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:05.737548 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1343/2661] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:52:05 2022] PUT /identity/v3/projects/fcc9f0aa45954a6284ac0f2f8f0e8403/users/41c73c7a9202466fa6db0933c4ba08c3/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:05.748410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337559b-63a4-4ad6-9094-f86cb25ffc88 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:05.748912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337559b-63a4-4ad6-9094-f86cb25ffc88 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:05.749303 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337559b-63a4-4ad6-9094-f86cb25ffc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:05.795845 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b337559b-63a4-4ad6-9094-f86cb25ffc88 None None] MFA Rules not processed for user `41c73c7a9202466fa6db0933c4ba08c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:05.797937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b337559b-63a4-4ad6-9094-f86cb25ffc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:05.899009 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1319/2662] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:52:05.911695 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e6fde-2e9e-4aa3-b994-401b11c082aa admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:05.914276 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3c9e6fde-2e9e-4aa3-b994-401b11c082aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:06.006602 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e6fde-2e9e-4aa3-b994-401b11c082aa admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:06.013057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d6f53-3739-4029-ba5d-9e29633187b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:06.013776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d6f53-3739-4029-ba5d-9e29633187b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:06.014057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d6f53-3739-4029-ba5d-9e29633187b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:06.014333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d6f53-3739-4029-ba5d-9e29633187b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:06.024815 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d6f53-3739-4029-ba5d-9e29633187b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:06.026350 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d6f53-3739-4029-ba5d-9e29633187b1 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:06.047372 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1344/2663] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:52:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:08.183928 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337559b-63a4-4ad6-9094-f86cb25ffc88 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:08.185926 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b337559b-63a4-4ad6-9094-f86cb25ffc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:08.305554 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337559b-63a4-4ad6-9094-f86cb25ffc88 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:08.313287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b29b32-fbad-4435-92c8-a4a99905529b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a0c1a4a6cf43e28ec1b8488c681848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:08.314050 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b29b32-fbad-4435-92c8-a4a99905529b service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:08.314386 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b29b32-fbad-4435-92c8-a4a99905529b service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:08.314768 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b29b32-fbad-4435-92c8-a4a99905529b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:08.317513 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-25b29b32-fbad-4435-92c8-a4a99905529b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:08.335911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b29b32-fbad-4435-92c8-a4a99905529b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:08.338105 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b29b32-fbad-4435-92c8-a4a99905529b service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:08.352375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1320/2664] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:52:08 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:52:15.136704 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d6f53-3739-4029-ba5d-9e29633187b1 service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:15.201088 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d6f53-3739-4029-ba5d-9e29633187b1 service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:15.205480 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b0ffe-5bde-4aeb-a7f0-190dde7c2731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:15.206115 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b0ffe-5bde-4aeb-a7f0-190dde7c2731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:15.206392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b0ffe-5bde-4aeb-a7f0-190dde7c2731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:15.206666 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b0ffe-5bde-4aeb-a7f0-190dde7c2731 admin admin] PATH_INFO: `/v3/users/41c73c7a9202466fa6db0933c4ba08c3` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:15.207808 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b0ffe-5bde-4aeb-a7f0-190dde7c2731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c73c7a9202466fa6db0933c4ba08c3)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:15.209755 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b0ffe-5bde-4aeb-a7f0-190dde7c2731 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:15.251127 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:15.259949 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ff5b0ffe-5bde-4aeb-a7f0-190dde7c2731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c73c7a9202466fa6db0933c4ba08c3'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:15.261558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ff5b0ffe-5bde-4aeb-a7f0-190dde7c2731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c73c7a9202466fa6db0933c4ba08c3'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:15.269164 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ff5b0ffe-5bde-4aeb-a7f0-190dde7c2731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c73c7a9202466fa6db0933c4ba08c3'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:15.282528 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ff5b0ffe-5bde-4aeb-a7f0-190dde7c2731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c73c7a9202466fa6db0933c4ba08c3'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:15.290180 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ff5b0ffe-5bde-4aeb-a7f0-190dde7c2731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c73c7a9202466fa6db0933c4ba08c3'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:15.296420 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1345/2665] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:52:15 2022] DELETE /identity/v3/users/41c73c7a9202466fa6db0933c4ba08c3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:15.300871 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3028e4-e4ac-49f5-95e0-fedba61e0e85 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:15.301375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3028e4-e4ac-49f5-95e0-fedba61e0e85 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:15.301472 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3028e4-e4ac-49f5-95e0-fedba61e0e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:15.314651 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:15.324165 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-eb3028e4-e4ac-49f5-95e0-fedba61e0e85 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:15.325481 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-eb3028e4-e4ac-49f5-95e0-fedba61e0e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:15.392363 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1321/2666] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:15.403691 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b0ffe-5bde-4aeb-a7f0-190dde7c2731 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:15.405079 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ff5b0ffe-5bde-4aeb-a7f0-190dde7c2731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:15.462363 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b0ffe-5bde-4aeb-a7f0-190dde7c2731 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:15.466571 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9162dc-dc6b-46f3-9ae1-be6396651c0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:15.467186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9162dc-dc6b-46f3-9ae1-be6396651c0e service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:15.467496 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9162dc-dc6b-46f3-9ae1-be6396651c0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:15.467767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9162dc-dc6b-46f3-9ae1-be6396651c0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:15.477005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9162dc-dc6b-46f3-9ae1-be6396651c0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:15.478450 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9162dc-dc6b-46f3-9ae1-be6396651c0e service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:15.488046 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1346/2667] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:52:15 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:15.783983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3028e4-e4ac-49f5-95e0-fedba61e0e85 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:15.785098 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-eb3028e4-e4ac-49f5-95e0-fedba61e0e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:15.800727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3028e4-e4ac-49f5-95e0-fedba61e0e85 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:15.806715 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14133118-2a04-4ac3-b5aa-6b4bc4128a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:15.809066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14133118-2a04-4ac3-b5aa-6b4bc4128a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:15.809425 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14133118-2a04-4ac3-b5aa-6b4bc4128a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:15.809687 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14133118-2a04-4ac3-b5aa-6b4bc4128a41 admin admin] PATH_INFO: `/v3/projects/fcc9f0aa45954a6284ac0f2f8f0e8403` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:15.811183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14133118-2a04-4ac3-b5aa-6b4bc4128a41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcc9f0aa45954a6284ac0f2f8f0e8403)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:15.813416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14133118-2a04-4ac3-b5aa-6b4bc4128a41 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:15.877428 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-14133118-2a04-4ac3-b5aa-6b4bc4128a41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcc9f0aa45954a6284ac0f2f8f0e8403', 'request_id': 'req-14133118-2a04-4ac3-b5aa-6b4bc4128a41'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:15.884098 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-14133118-2a04-4ac3-b5aa-6b4bc4128a41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcc9f0aa45954a6284ac0f2f8f0e8403', 'request_id': 'req-14133118-2a04-4ac3-b5aa-6b4bc4128a41'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:15.893704 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-14133118-2a04-4ac3-b5aa-6b4bc4128a41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcc9f0aa45954a6284ac0f2f8f0e8403', 'request_id': 'req-14133118-2a04-4ac3-b5aa-6b4bc4128a41'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:15.908275 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-14133118-2a04-4ac3-b5aa-6b4bc4128a41 admin admin] The token cache is being invalidate because project fcc9f0aa45954a6284ac0f2f8f0e8403 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:52:15.908574 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-14133118-2a04-4ac3-b5aa-6b4bc4128a41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:15.909268 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-14133118-2a04-4ac3-b5aa-6b4bc4128a41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:15.910794 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1322/2668] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:52:15 2022] DELETE /identity/v3/projects/fcc9f0aa45954a6284ac0f2f8f0e8403 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:15.915852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e1287-1164-41b6-b6ea-38c23467ec90 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:15.916230 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e1287-1164-41b6-b6ea-38c23467ec90 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:15.916681 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e1287-1164-41b6-b6ea-38c23467ec90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:15.939812 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-776e1287-1164-41b6-b6ea-38c23467ec90 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:15.941045 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-776e1287-1164-41b6-b6ea-38c23467ec90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:16.019956 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1347/2669] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:16.031482 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3848a09-0c5e-40a6-8dd5-db88eb2f2cc3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:16.032117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3848a09-0c5e-40a6-8dd5-db88eb2f2cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:16.032563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3848a09-0c5e-40a6-8dd5-db88eb2f2cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:16.056143 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e3848a09-0c5e-40a6-8dd5-db88eb2f2cc3 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:16.057987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e3848a09-0c5e-40a6-8dd5-db88eb2f2cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:16.066169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1323/2670] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:16.070612 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e1287-1164-41b6-b6ea-38c23467ec90 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:16.080861 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e1287-1164-41b6-b6ea-38c23467ec90 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:16.086011 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e7de6-63df-41d4-8bfe-f7031eabcae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:16.086741 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e7de6-63df-41d4-8bfe-f7031eabcae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:16.087179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e7de6-63df-41d4-8bfe-f7031eabcae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:16.087562 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e7de6-63df-41d4-8bfe-f7031eabcae1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:16.088433 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e7de6-63df-41d4-8bfe-f7031eabcae1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:52:16.088852 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e7de6-63df-41d4-8bfe-f7031eabcae1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:16.090647 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e7de6-63df-41d4-8bfe-f7031eabcae1 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:16.105101 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1348/2671] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:52:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:52:16.110229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3848a09-0c5e-40a6-8dd5-db88eb2f2cc3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:16.120890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3848a09-0c5e-40a6-8dd5-db88eb2f2cc3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:16.128948 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c712933-a1f8-4e58-8ef7-4e5b5573f952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:16.129880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c712933-a1f8-4e58-8ef7-4e5b5573f952 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:16.130346 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c712933-a1f8-4e58-8ef7-4e5b5573f952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:16.130627 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c712933-a1f8-4e58-8ef7-4e5b5573f952 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:16.131441 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c712933-a1f8-4e58-8ef7-4e5b5573f952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:16.134640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c712933-a1f8-4e58-8ef7-4e5b5573f952 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:16.171382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1324/2672] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:52:16 2022] POST /identity/v3/projects => generated 378 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:52:16.175372 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e7de6-63df-41d4-8bfe-f7031eabcae1 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:16.187641 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e7de6-63df-41d4-8bfe-f7031eabcae1 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:16.234696 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:16.252902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673cade-4e98-43d2-a46c-2fe0a76ecc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:16.253920 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673cade-4e98-43d2-a46c-2fe0a76ecc99 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:16.254231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673cade-4e98-43d2-a46c-2fe0a76ecc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:16.254505 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673cade-4e98-43d2-a46c-2fe0a76ecc99 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:16.255178 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673cade-4e98-43d2-a46c-2fe0a76ecc99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:16.257162 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673cade-4e98-43d2-a46c-2fe0a76ecc99 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:16.281833 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1349/2673] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:52:16 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:52:16.286559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c712933-a1f8-4e58-8ef7-4e5b5573f952 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:16.298300 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c712933-a1f8-4e58-8ef7-4e5b5573f952 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:16.303087 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595ffa3-99b9-4bc3-8443-f7beb026465c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:16.303956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595ffa3-99b9-4bc3-8443-f7beb026465c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:16.304068 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595ffa3-99b9-4bc3-8443-f7beb026465c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:16.304253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595ffa3-99b9-4bc3-8443-f7beb026465c admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:16.304949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595ffa3-99b9-4bc3-8443-f7beb026465c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:16.306582 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595ffa3-99b9-4bc3-8443-f7beb026465c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:16.318998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1325/2674] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:16 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:52:16.324429 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673cade-4e98-43d2-a46c-2fe0a76ecc99 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:16.334914 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673cade-4e98-43d2-a46c-2fe0a76ecc99 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:16.340348 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b4233-b1d6-4f62-b2b3-02f9741989cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:16.340963 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b4233-b1d6-4f62-b2b3-02f9741989cb admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:16.341291 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b4233-b1d6-4f62-b2b3-02f9741989cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:16.341573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b4233-b1d6-4f62-b2b3-02f9741989cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:16.342272 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b4233-b1d6-4f62-b2b3-02f9741989cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:16.344553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b4233-b1d6-4f62-b2b3-02f9741989cb admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:16.356892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1350/2675] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:16 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:52:16.361292 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595ffa3-99b9-4bc3-8443-f7beb026465c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:16.372226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595ffa3-99b9-4bc3-8443-f7beb026465c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:16.376986 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72beb8c-325d-4347-be10-51108c218b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:16.377698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72beb8c-325d-4347-be10-51108c218b6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:16.377983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72beb8c-325d-4347-be10-51108c218b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:16.378265 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72beb8c-325d-4347-be10-51108c218b6a admin admin] PATH_INFO: `/v3/projects/f475861417de49a3900250b1a4966f7a/users/cd4ad86442d44c2294967370b2fd0bce/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:16.394943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72beb8c-325d-4347-be10-51108c218b6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f475861417de49a3900250b1a4966f7a, user_id=cd4ad86442d44c2294967370b2fd0bce, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:16.397700 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72beb8c-325d-4347-be10-51108c218b6a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:16.412769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1326/2676] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:52:16 2022] PUT /identity/v3/projects/f475861417de49a3900250b1a4966f7a/users/cd4ad86442d44c2294967370b2fd0bce/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:16.423508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b7ece-500b-413a-bdd2-d4e2c5421fc6 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:16.423794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b7ece-500b-413a-bdd2-d4e2c5421fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:16.424069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b7ece-500b-413a-bdd2-d4e2c5421fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:16.455124 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-b98b7ece-500b-413a-bdd2-d4e2c5421fc6 None None] MFA Rules not processed for user `cd4ad86442d44c2294967370b2fd0bce`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:16.456359 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-b98b7ece-500b-413a-bdd2-d4e2c5421fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:16.535281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1351/2677] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:52:16.539753 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72beb8c-325d-4347-be10-51108c218b6a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:16.579111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:16.614479 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72beb8c-325d-4347-be10-51108c218b6a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:16.622015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f044ffbe-41fc-421b-879a-4d1a6d6d0706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:16.622653 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f044ffbe-41fc-421b-879a-4d1a6d6d0706 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:16.622937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f044ffbe-41fc-421b-879a-4d1a6d6d0706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:16.623216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f044ffbe-41fc-421b-879a-4d1a6d6d0706 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:16.623890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f044ffbe-41fc-421b-879a-4d1a6d6d0706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:16.625827 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f044ffbe-41fc-421b-879a-4d1a6d6d0706 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:16.655385 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1327/2678] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:52:16 2022] POST /identity/v3/projects => generated 380 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:52:16.660204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b7ece-500b-413a-bdd2-d4e2c5421fc6 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:16.672177 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b7ece-500b-413a-bdd2-d4e2c5421fc6 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:16.740377 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45260bd-dd4e-4d36-8683-1876def86098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:16.740997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45260bd-dd4e-4d36-8683-1876def86098 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:16.741315 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45260bd-dd4e-4d36-8683-1876def86098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:16.741593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45260bd-dd4e-4d36-8683-1876def86098 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:16.742274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45260bd-dd4e-4d36-8683-1876def86098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:16.744207 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45260bd-dd4e-4d36-8683-1876def86098 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:16.768948 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1352/2679] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:52:16 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:52:16.773196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f044ffbe-41fc-421b-879a-4d1a6d6d0706 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:16.784342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f044ffbe-41fc-421b-879a-4d1a6d6d0706 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:16.789826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a91d40-50cb-4730-95f0-8c4babdad8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:16.790402 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a91d40-50cb-4730-95f0-8c4babdad8b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:16.790687 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a91d40-50cb-4730-95f0-8c4babdad8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:16.791131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a91d40-50cb-4730-95f0-8c4babdad8b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:16.792492 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a91d40-50cb-4730-95f0-8c4babdad8b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:16.794106 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a91d40-50cb-4730-95f0-8c4babdad8b0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:16.808605 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1328/2680] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:16 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:52:16.813675 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45260bd-dd4e-4d36-8683-1876def86098 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:16.824602 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45260bd-dd4e-4d36-8683-1876def86098 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:16.829682 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ce226-d095-441f-9e2f-47b2f34d68ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:16.830314 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ce226-d095-441f-9e2f-47b2f34d68ef admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:16.830595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ce226-d095-441f-9e2f-47b2f34d68ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:16.830872 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ce226-d095-441f-9e2f-47b2f34d68ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:16.831642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ce226-d095-441f-9e2f-47b2f34d68ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:16.833343 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ce226-d095-441f-9e2f-47b2f34d68ef admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:16.844596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1353/2681] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:16 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:52:16.848825 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a91d40-50cb-4730-95f0-8c4babdad8b0 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:16.860909 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a91d40-50cb-4730-95f0-8c4babdad8b0 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:16.865718 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda7d1d-2447-45e6-8b3b-6fcf8d93d766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:16.866396 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda7d1d-2447-45e6-8b3b-6fcf8d93d766 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:16.866674 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda7d1d-2447-45e6-8b3b-6fcf8d93d766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:16.866949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda7d1d-2447-45e6-8b3b-6fcf8d93d766 admin admin] PATH_INFO: `/v3/projects/46e69845d7254a59a652d92a0f3f2529/users/084b9c8f71344049bd17747513f3caa2/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:16.883660 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda7d1d-2447-45e6-8b3b-6fcf8d93d766 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46e69845d7254a59a652d92a0f3f2529, user_id=084b9c8f71344049bd17747513f3caa2, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:16.890172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda7d1d-2447-45e6-8b3b-6fcf8d93d766 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:16.904952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1329/2682] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:52:16 2022] PUT /identity/v3/projects/46e69845d7254a59a652d92a0f3f2529/users/084b9c8f71344049bd17747513f3caa2/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:16.918374 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc9bc9-407a-4023-9eb6-37b67d63a508 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:16.918894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc9bc9-407a-4023-9eb6-37b67d63a508 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:16.919175 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc9bc9-407a-4023-9eb6-37b67d63a508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:16.955201 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-a0dc9bc9-407a-4023-9eb6-37b67d63a508 None None] MFA Rules not processed for user `084b9c8f71344049bd17747513f3caa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:16.956437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-a0dc9bc9-407a-4023-9eb6-37b67d63a508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:16.978655 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:17.031707 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1354/2683] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:16 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:17.038139 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda7d1d-2447-45e6-8b3b-6fcf8d93d766 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:17.111119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda7d1d-2447-45e6-8b3b-6fcf8d93d766 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:17.116911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5f6d1-e975-4b56-94ee-99363e340f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:17.117576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5f6d1-e975-4b56-94ee-99363e340f7c admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:17.117912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5f6d1-e975-4b56-94ee-99363e340f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:17.118202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5f6d1-e975-4b56-94ee-99363e340f7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:17.118934 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5f6d1-e975-4b56-94ee-99363e340f7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:17.121241 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5f6d1-e975-4b56-94ee-99363e340f7c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:17.155362 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1330/2684] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:52:17 2022] POST /identity/v3/projects => generated 380 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:52:17.159999 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc9bc9-407a-4023-9eb6-37b67d63a508 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:17.172136 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc9bc9-407a-4023-9eb6-37b67d63a508 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:17.237233 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f881d-2ace-41b1-88dc-86decb3195ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:17.237880 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f881d-2ace-41b1-88dc-86decb3195ca admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:17.238127 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f881d-2ace-41b1-88dc-86decb3195ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:17.238399 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f881d-2ace-41b1-88dc-86decb3195ca admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:17.239062 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f881d-2ace-41b1-88dc-86decb3195ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:17.240996 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f881d-2ace-41b1-88dc-86decb3195ca admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:17.270003 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1355/2685] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:52:17 2022] POST /identity/v3/users => generated 297 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:52:17.274325 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5f6d1-e975-4b56-94ee-99363e340f7c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:17.286381 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5f6d1-e975-4b56-94ee-99363e340f7c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:17.291169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8531f-11b6-4680-814a-0dbaa29c2772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:17.292421 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8531f-11b6-4680-814a-0dbaa29c2772 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:17.292722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8531f-11b6-4680-814a-0dbaa29c2772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:17.293038 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8531f-11b6-4680-814a-0dbaa29c2772 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:17.293937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8531f-11b6-4680-814a-0dbaa29c2772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:17.295538 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8531f-11b6-4680-814a-0dbaa29c2772 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:17.307424 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1331/2686] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:17 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:52:17.313005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f881d-2ace-41b1-88dc-86decb3195ca admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:17.324684 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f881d-2ace-41b1-88dc-86decb3195ca admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:17.329878 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a88d5-d81b-496d-a2b1-9f0b4e7dabfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:17.330559 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a88d5-d81b-496d-a2b1-9f0b4e7dabfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:17.330841 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a88d5-d81b-496d-a2b1-9f0b4e7dabfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:17.331115 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a88d5-d81b-496d-a2b1-9f0b4e7dabfa admin admin] PATH_INFO: `/v3/projects/3ef3937a4dd148a6a7d77931d80a04d5/users/9ede163a5e024d21bd18697b6489c748/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:17.348162 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a88d5-d81b-496d-a2b1-9f0b4e7dabfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ef3937a4dd148a6a7d77931d80a04d5, user_id=9ede163a5e024d21bd18697b6489c748, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:17.351213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a88d5-d81b-496d-a2b1-9f0b4e7dabfa admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:17.369387 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1356/2687] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:52:17 2022] PUT /identity/v3/projects/3ef3937a4dd148a6a7d77931d80a04d5/users/9ede163a5e024d21bd18697b6489c748/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:52:17.373601 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8531f-11b6-4680-814a-0dbaa29c2772 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:17.384205 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8531f-11b6-4680-814a-0dbaa29c2772 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:17.434592 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:17.448305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67664b6-5f8b-4e96-b8f8-f0dcc9a13977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:17.448941 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67664b6-5f8b-4e96-b8f8-f0dcc9a13977 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:17.449252 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67664b6-5f8b-4e96-b8f8-f0dcc9a13977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:17.449528 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67664b6-5f8b-4e96-b8f8-f0dcc9a13977 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:17.450223 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67664b6-5f8b-4e96-b8f8-f0dcc9a13977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:17.451815 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67664b6-5f8b-4e96-b8f8-f0dcc9a13977 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:17.468702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1332/2688] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:17 2022] GET /identity/v3/roles => generated 1427 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:52:17.473317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a88d5-d81b-496d-a2b1-9f0b4e7dabfa admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:17.484584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a88d5-d81b-496d-a2b1-9f0b4e7dabfa admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:17.489328 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52a1f7-b88f-432c-ae99-0638b94aeb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:17.490006 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52a1f7-b88f-432c-ae99-0638b94aeb5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:17.490294 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52a1f7-b88f-432c-ae99-0638b94aeb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:17.490570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52a1f7-b88f-432c-ae99-0638b94aeb5c admin admin] PATH_INFO: `/v3/projects/3ef3937a4dd148a6a7d77931d80a04d5/users/9ede163a5e024d21bd18697b6489c748/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:17.492578 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f52a1f7-b88f-432c-ae99-0638b94aeb5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ef3937a4dd148a6a7d77931d80a04d5, user_id=9ede163a5e024d21bd18697b6489c748, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:17.495986 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f52a1f7-b88f-432c-ae99-0638b94aeb5c admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:17.507520 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1357/2689] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:52:17 2022] PUT /identity/v3/projects/3ef3937a4dd148a6a7d77931d80a04d5/users/9ede163a5e024d21bd18697b6489c748/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:52:17.518621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41dd843-779c-498c-846b-24d15d7931b2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:17.518911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41dd843-779c-498c-846b-24d15d7931b2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:17.519205 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41dd843-779c-498c-846b-24d15d7931b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:17.552429 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b41dd843-779c-498c-846b-24d15d7931b2 None None] MFA Rules not processed for user `9ede163a5e024d21bd18697b6489c748`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:17.553728 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b41dd843-779c-498c-846b-24d15d7931b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:17.630903 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1333/2690] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:17 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:52:17.643570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52a1f7-b88f-432c-ae99-0638b94aeb5c admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:17.645201 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-6f52a1f7-b88f-432c-ae99-0638b94aeb5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:17.711836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52a1f7-b88f-432c-ae99-0638b94aeb5c admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:17.716703 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d08ae-d5c5-47b5-91bc-1223637310ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:17.717352 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d08ae-d5c5-47b5-91bc-1223637310ee service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:17.717637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d08ae-d5c5-47b5-91bc-1223637310ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:17.717914 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d08ae-d5c5-47b5-91bc-1223637310ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:17.727916 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d08ae-d5c5-47b5-91bc-1223637310ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:17.729587 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d08ae-d5c5-47b5-91bc-1223637310ee service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:17.739652 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1358/2691] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:52:17 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:17.785275 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41dd843-779c-498c-846b-24d15d7931b2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:17.786752 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b41dd843-779c-498c-846b-24d15d7931b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:17.799533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41dd843-779c-498c-846b-24d15d7931b2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:17.803193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03a5e4-ca3c-4776-8ac2-cc078f9ac341 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:17.803804 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03a5e4-ca3c-4776-8ac2-cc078f9ac341 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:17.804142 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03a5e4-ca3c-4776-8ac2-cc078f9ac341 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:17.804371 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03a5e4-ca3c-4776-8ac2-cc078f9ac341 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:17.869336 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03a5e4-ca3c-4776-8ac2-cc078f9ac341 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:17.871731 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03a5e4-ca3c-4776-8ac2-cc078f9ac341 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:17.881765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1334/2692] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:52:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:21.011280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d08ae-d5c5-47b5-91bc-1223637310ee service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:21.036819 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:21.105179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d08ae-d5c5-47b5-91bc-1223637310ee service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:21.110619 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f2f90-5737-4680-bc3e-47513054dc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:21.111380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f2f90-5737-4680-bc3e-47513054dc72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:21.111700 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f2f90-5737-4680-bc3e-47513054dc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:21.112014 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f2f90-5737-4680-bc3e-47513054dc72 admin admin] PATH_INFO: `/v3/users/cd4ad86442d44c2294967370b2fd0bce` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:21.113302 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f2f90-5737-4680-bc3e-47513054dc72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd4ad86442d44c2294967370b2fd0bce)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:21.115393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f2f90-5737-4680-bc3e-47513054dc72 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:21.174520 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-1f8f2f90-5737-4680-bc3e-47513054dc72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd4ad86442d44c2294967370b2fd0bce'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:21.175113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-1f8f2f90-5737-4680-bc3e-47513054dc72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd4ad86442d44c2294967370b2fd0bce'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:21.184657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-1f8f2f90-5737-4680-bc3e-47513054dc72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd4ad86442d44c2294967370b2fd0bce'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:21.199627 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-1f8f2f90-5737-4680-bc3e-47513054dc72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd4ad86442d44c2294967370b2fd0bce'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:21.206785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-1f8f2f90-5737-4680-bc3e-47513054dc72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd4ad86442d44c2294967370b2fd0bce'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:21.213021 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1359/2693] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:52:21 2022] DELETE /identity/v3/users/cd4ad86442d44c2294967370b2fd0bce => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:21.218164 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d542e9-ac3d-426c-a23c-d93aae9f9fce None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:21.221437 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d542e9-ac3d-426c-a23c-d93aae9f9fce None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:21.221437 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d542e9-ac3d-426c-a23c-d93aae9f9fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:21.239619 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c4d542e9-ac3d-426c-a23c-d93aae9f9fce None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:21.240834 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c4d542e9-ac3d-426c-a23c-d93aae9f9fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:21.315169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1335/2694] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:21.326437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f2f90-5737-4680-bc3e-47513054dc72 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:21.327888 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1f8f2f90-5737-4680-bc3e-47513054dc72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:21.392903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f2f90-5737-4680-bc3e-47513054dc72 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:21.396870 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d071336-2dea-4248-96f5-4d4f14b2f89e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:21.397521 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d071336-2dea-4248-96f5-4d4f14b2f89e service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:21.397797 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d071336-2dea-4248-96f5-4d4f14b2f89e service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:21.398067 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d071336-2dea-4248-96f5-4d4f14b2f89e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:21.409479 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d071336-2dea-4248-96f5-4d4f14b2f89e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:21.411029 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d071336-2dea-4248-96f5-4d4f14b2f89e service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:21.422224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1360/2695] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:52:21 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:21.756641 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d542e9-ac3d-426c-a23c-d93aae9f9fce None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:21.758743 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c4d542e9-ac3d-426c-a23c-d93aae9f9fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:21.780991 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d542e9-ac3d-426c-a23c-d93aae9f9fce None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:21.786401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5dd91-1f1a-4be2-97b0-221e79108c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:21.787130 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5dd91-1f1a-4be2-97b0-221e79108c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:21.787427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5dd91-1f1a-4be2-97b0-221e79108c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:21.787713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5dd91-1f1a-4be2-97b0-221e79108c2f admin admin] PATH_INFO: `/v3/projects/f475861417de49a3900250b1a4966f7a` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:21.789196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5dd91-1f1a-4be2-97b0-221e79108c2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f475861417de49a3900250b1a4966f7a)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:21.791823 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5dd91-1f1a-4be2-97b0-221e79108c2f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:21.802386 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:21.862672 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8de5dd91-1f1a-4be2-97b0-221e79108c2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f475861417de49a3900250b1a4966f7a', 'request_id': 'req-8de5dd91-1f1a-4be2-97b0-221e79108c2f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:21.873064 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8de5dd91-1f1a-4be2-97b0-221e79108c2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f475861417de49a3900250b1a4966f7a', 'request_id': 'req-8de5dd91-1f1a-4be2-97b0-221e79108c2f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:21.886529 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8de5dd91-1f1a-4be2-97b0-221e79108c2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f475861417de49a3900250b1a4966f7a', 'request_id': 'req-8de5dd91-1f1a-4be2-97b0-221e79108c2f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:21.898876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8de5dd91-1f1a-4be2-97b0-221e79108c2f admin admin] The token cache is being invalidate because project f475861417de49a3900250b1a4966f7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:52:21.899096 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8de5dd91-1f1a-4be2-97b0-221e79108c2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:21.899818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8de5dd91-1f1a-4be2-97b0-221e79108c2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:21.901895 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1336/2696] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:52:21 2022] DELETE /identity/v3/projects/f475861417de49a3900250b1a4966f7a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:21.906553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d071336-2dea-4248-96f5-4d4f14b2f89e service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:21.918228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d071336-2dea-4248-96f5-4d4f14b2f89e service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:21.975083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:22.002376 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28302b6-c302-48b5-a5a4-39902ec2b9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:22.003255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28302b6-c302-48b5-a5a4-39902ec2b9fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:22.004065 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28302b6-c302-48b5-a5a4-39902ec2b9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:22.004265 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28302b6-c302-48b5-a5a4-39902ec2b9fd admin admin] PATH_INFO: `/v3/users/084b9c8f71344049bd17747513f3caa2` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:22.005948 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28302b6-c302-48b5-a5a4-39902ec2b9fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=084b9c8f71344049bd17747513f3caa2)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:22.008018 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28302b6-c302-48b5-a5a4-39902ec2b9fd admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:22.082450 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-e28302b6-c302-48b5-a5a4-39902ec2b9fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '084b9c8f71344049bd17747513f3caa2'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:22.083444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-e28302b6-c302-48b5-a5a4-39902ec2b9fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '084b9c8f71344049bd17747513f3caa2'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:22.094580 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-e28302b6-c302-48b5-a5a4-39902ec2b9fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '084b9c8f71344049bd17747513f3caa2'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:22.111437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-e28302b6-c302-48b5-a5a4-39902ec2b9fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '084b9c8f71344049bd17747513f3caa2'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:22.125831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-e28302b6-c302-48b5-a5a4-39902ec2b9fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '084b9c8f71344049bd17747513f3caa2'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:22.137919 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1361/2697] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:52:21 2022] DELETE /identity/v3/users/084b9c8f71344049bd17747513f3caa2 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:22.527757 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5dd91-1f1a-4be2-97b0-221e79108c2f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:22.529315 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8de5dd91-1f1a-4be2-97b0-221e79108c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:22.545906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5dd91-1f1a-4be2-97b0-221e79108c2f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:22.626199 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce1886-584f-469b-ba08-a10bea909d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:22.626883 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce1886-584f-469b-ba08-a10bea909d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:22.627367 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce1886-584f-469b-ba08-a10bea909d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:22.627658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce1886-584f-469b-ba08-a10bea909d29 admin admin] PATH_INFO: `/v3/projects/46e69845d7254a59a652d92a0f3f2529` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:22.629290 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce1886-584f-469b-ba08-a10bea909d29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46e69845d7254a59a652d92a0f3f2529)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:22.631285 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce1886-584f-469b-ba08-a10bea909d29 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:22.706536 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dfce1886-584f-469b-ba08-a10bea909d29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46e69845d7254a59a652d92a0f3f2529', 'request_id': 'req-dfce1886-584f-469b-ba08-a10bea909d29'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:22.718695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dfce1886-584f-469b-ba08-a10bea909d29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46e69845d7254a59a652d92a0f3f2529', 'request_id': 'req-dfce1886-584f-469b-ba08-a10bea909d29'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:22.731275 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dfce1886-584f-469b-ba08-a10bea909d29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46e69845d7254a59a652d92a0f3f2529', 'request_id': 'req-dfce1886-584f-469b-ba08-a10bea909d29'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:22.746688 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dfce1886-584f-469b-ba08-a10bea909d29 admin admin] The token cache is being invalidate because project 46e69845d7254a59a652d92a0f3f2529 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:52:22.747059 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dfce1886-584f-469b-ba08-a10bea909d29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:22.747921 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dfce1886-584f-469b-ba08-a10bea909d29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:22.749995 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1337/2698] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:52:22 2022] DELETE /identity/v3/projects/46e69845d7254a59a652d92a0f3f2529 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:22.756337 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28302b6-c302-48b5-a5a4-39902ec2b9fd admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:22.758590 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e28302b6-c302-48b5-a5a4-39902ec2b9fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:22.778851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28302b6-c302-48b5-a5a4-39902ec2b9fd admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:22.870814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24da4d-ab35-475f-bf4d-2ace523df640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:22.871469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24da4d-ab35-475f-bf4d-2ace523df640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:22.871864 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24da4d-ab35-475f-bf4d-2ace523df640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:22.872139 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24da4d-ab35-475f-bf4d-2ace523df640 admin admin] PATH_INFO: `/v3/users/9ede163a5e024d21bd18697b6489c748` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:22.873389 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c24da4d-ab35-475f-bf4d-2ace523df640 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ede163a5e024d21bd18697b6489c748)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:22.875358 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c24da4d-ab35-475f-bf4d-2ace523df640 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:22.949543 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-9c24da4d-ab35-475f-bf4d-2ace523df640 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ede163a5e024d21bd18697b6489c748'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:22.950654 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-9c24da4d-ab35-475f-bf4d-2ace523df640 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ede163a5e024d21bd18697b6489c748'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:22.963016 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-9c24da4d-ab35-475f-bf4d-2ace523df640 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ede163a5e024d21bd18697b6489c748'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:22.976268 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:22.990696 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-9c24da4d-ab35-475f-bf4d-2ace523df640 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ede163a5e024d21bd18697b6489c748'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:23.003992 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-9c24da4d-ab35-475f-bf4d-2ace523df640 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ede163a5e024d21bd18697b6489c748'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:23.015609 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1362/2699] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:52:22 2022] DELETE /identity/v3/users/9ede163a5e024d21bd18697b6489c748 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:23.383658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce1886-584f-469b-ba08-a10bea909d29 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:23.385458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-dfce1886-584f-469b-ba08-a10bea909d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:23.401878 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce1886-584f-469b-ba08-a10bea909d29 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:23.428401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:23.489732 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0e6d2-b7a4-4956-a033-1ca9d261e123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:23.491423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0e6d2-b7a4-4956-a033-1ca9d261e123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:23.491423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0e6d2-b7a4-4956-a033-1ca9d261e123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:23.491423 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0e6d2-b7a4-4956-a033-1ca9d261e123 admin admin] PATH_INFO: `/v3/projects/3ef3937a4dd148a6a7d77931d80a04d5` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:23.492597 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0e6d2-b7a4-4956-a033-1ca9d261e123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ef3937a4dd148a6a7d77931d80a04d5)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:23.495362 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0e6d2-b7a4-4956-a033-1ca9d261e123 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:23.578834 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-87e0e6d2-b7a4-4956-a033-1ca9d261e123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ef3937a4dd148a6a7d77931d80a04d5', 'request_id': 'req-87e0e6d2-b7a4-4956-a033-1ca9d261e123'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:23.585968 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-87e0e6d2-b7a4-4956-a033-1ca9d261e123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ef3937a4dd148a6a7d77931d80a04d5', 'request_id': 'req-87e0e6d2-b7a4-4956-a033-1ca9d261e123'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:23.595936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-87e0e6d2-b7a4-4956-a033-1ca9d261e123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ef3937a4dd148a6a7d77931d80a04d5', 'request_id': 'req-87e0e6d2-b7a4-4956-a033-1ca9d261e123'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:23.610831 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-87e0e6d2-b7a4-4956-a033-1ca9d261e123 admin admin] The token cache is being invalidate because project 3ef3937a4dd148a6a7d77931d80a04d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:52:23.611233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-87e0e6d2-b7a4-4956-a033-1ca9d261e123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:23.612043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-87e0e6d2-b7a4-4956-a033-1ca9d261e123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:23.613940 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1338/2700] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:52:23 2022] DELETE /identity/v3/projects/3ef3937a4dd148a6a7d77931d80a04d5 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:23.620450 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc368b5c-33bb-4c71-b3ec-d08dc2f77359 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:23.620967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc368b5c-33bb-4c71-b3ec-d08dc2f77359 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:23.621083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc368b5c-33bb-4c71-b3ec-d08dc2f77359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:23.645440 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-fc368b5c-33bb-4c71-b3ec-d08dc2f77359 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:23.646590 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-fc368b5c-33bb-4c71-b3ec-d08dc2f77359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:23.734255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1363/2701] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:23.747659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271aba18-8949-4a2e-9582-b0cc7f46188c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:23.747946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271aba18-8949-4a2e-9582-b0cc7f46188c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:23.748229 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271aba18-8949-4a2e-9582-b0cc7f46188c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:23.777444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-271aba18-8949-4a2e-9582-b0cc7f46188c None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:23.779124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-271aba18-8949-4a2e-9582-b0cc7f46188c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:23.787981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1339/2702] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:23.793506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc368b5c-33bb-4c71-b3ec-d08dc2f77359 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:23.811241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc368b5c-33bb-4c71-b3ec-d08dc2f77359 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:23.818483 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac32153-85e4-4f9b-a571-1777c3b3645c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:23.819382 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac32153-85e4-4f9b-a571-1777c3b3645c admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:23.819701 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac32153-85e4-4f9b-a571-1777c3b3645c admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:23.819974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac32153-85e4-4f9b-a571-1777c3b3645c admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:23.820717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac32153-85e4-4f9b-a571-1777c3b3645c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:52:23.821029 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac32153-85e4-4f9b-a571-1777c3b3645c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:23.823392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac32153-85e4-4f9b-a571-1777c3b3645c admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:23.838926 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1364/2703] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:52:23 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:52:23.844183 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271aba18-8949-4a2e-9582-b0cc7f46188c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:23.860774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271aba18-8949-4a2e-9582-b0cc7f46188c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:23.868188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f75e5-a3ae-437f-82bf-951a9869a236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:23.869757 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f75e5-a3ae-437f-82bf-951a9869a236 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:23.870238 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f75e5-a3ae-437f-82bf-951a9869a236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:23.870702 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f75e5-a3ae-437f-82bf-951a9869a236 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:23.871902 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f75e5-a3ae-437f-82bf-951a9869a236 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:23.875507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f75e5-a3ae-437f-82bf-951a9869a236 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:23.916398 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1340/2704] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:52:23 2022] POST /identity/v3/projects => generated 366 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:52:23.919815 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac32153-85e4-4f9b-a571-1777c3b3645c admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:23.935299 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac32153-85e4-4f9b-a571-1777c3b3645c admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:24.013327 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954520f-4eba-41ad-9c1c-96e53ebd5749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:24.013966 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954520f-4eba-41ad-9c1c-96e53ebd5749 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:24.014251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954520f-4eba-41ad-9c1c-96e53ebd5749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:24.014531 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954520f-4eba-41ad-9c1c-96e53ebd5749 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:24.015206 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954520f-4eba-41ad-9c1c-96e53ebd5749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:24.017315 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954520f-4eba-41ad-9c1c-96e53ebd5749 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:24.049302 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1365/2705] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:52:23 2022] POST /identity/v3/users => generated 284 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:52:24.054219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f75e5-a3ae-437f-82bf-951a9869a236 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:24.067251 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f75e5-a3ae-437f-82bf-951a9869a236 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:24.074800 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed66f4-f957-43ef-97a1-0a926bb5ad91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:24.075750 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed66f4-f957-43ef-97a1-0a926bb5ad91 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:24.076055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed66f4-f957-43ef-97a1-0a926bb5ad91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:24.076360 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed66f4-f957-43ef-97a1-0a926bb5ad91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:24.077552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed66f4-f957-43ef-97a1-0a926bb5ad91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:24.080762 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed66f4-f957-43ef-97a1-0a926bb5ad91 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:24.100454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1341/2706] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:24 2022] GET /identity/v3/roles => generated 1427 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:52:24.106572 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954520f-4eba-41ad-9c1c-96e53ebd5749 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:24.122387 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954520f-4eba-41ad-9c1c-96e53ebd5749 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:24.128151 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c9808-94d7-4093-9394-e87bb362b9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:24.129062 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c9808-94d7-4093-9394-e87bb362b9f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:24.129382 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c9808-94d7-4093-9394-e87bb362b9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:24.129658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c9808-94d7-4093-9394-e87bb362b9f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:24.130375 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c9808-94d7-4093-9394-e87bb362b9f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:24.132164 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c9808-94d7-4093-9394-e87bb362b9f7 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:24.157216 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1366/2707] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:24 2022] GET /identity/v3/roles => generated 1427 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:52:24.162052 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed66f4-f957-43ef-97a1-0a926bb5ad91 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:24.174388 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed66f4-f957-43ef-97a1-0a926bb5ad91 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:24.179874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d4b70-2005-4cd9-8878-e548bda35ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:24.180691 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d4b70-2005-4cd9-8878-e548bda35ce1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:24.181021 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d4b70-2005-4cd9-8878-e548bda35ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:24.181383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d4b70-2005-4cd9-8878-e548bda35ce1 admin admin] PATH_INFO: `/v3/projects/a216e62e78e147ff8b5d6c37eeee30a5/users/c6258cdbae5e410db4a2ba90b9be9da8/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:24.201491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d4b70-2005-4cd9-8878-e548bda35ce1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a216e62e78e147ff8b5d6c37eeee30a5, user_id=c6258cdbae5e410db4a2ba90b9be9da8, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:24.204193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d4b70-2005-4cd9-8878-e548bda35ce1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:24.222998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1342/2708] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:52:24 2022] PUT /identity/v3/projects/a216e62e78e147ff8b5d6c37eeee30a5/users/c6258cdbae5e410db4a2ba90b9be9da8/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:52:24.243859 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cc326-7058-4669-a5a0-4ed7a53223f9 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:24.244190 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cc326-7058-4669-a5a0-4ed7a53223f9 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:24.244798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cc326-7058-4669-a5a0-4ed7a53223f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:24.282727 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-3a4cc326-7058-4669-a5a0-4ed7a53223f9 None None] MFA Rules not processed for user `c6258cdbae5e410db4a2ba90b9be9da8`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:24.284698 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3a4cc326-7058-4669-a5a0-4ed7a53223f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:24.323981 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:24.370912 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1367/2709] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:52:24.376010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d4b70-2005-4cd9-8878-e548bda35ce1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:24.474205 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:24.491176 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d4b70-2005-4cd9-8878-e548bda35ce1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:24.500563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f89b31-8fa6-4b34-a2a8-a8fb83cd9e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:24.501704 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f89b31-8fa6-4b34-a2a8-a8fb83cd9e0e admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:24.502204 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f89b31-8fa6-4b34-a2a8-a8fb83cd9e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:24.502677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f89b31-8fa6-4b34-a2a8-a8fb83cd9e0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:24.504138 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f89b31-8fa6-4b34-a2a8-a8fb83cd9e0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:24.508016 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f89b31-8fa6-4b34-a2a8-a8fb83cd9e0e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:24.553543 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1343/2710] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:52:24 2022] POST /identity/v3/projects => generated 364 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:52:24.558551 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cc326-7058-4669-a5a0-4ed7a53223f9 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:24.570369 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cc326-7058-4669-a5a0-4ed7a53223f9 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:24.643841 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5413c6-1195-4cc0-bd0d-d7b1cf739bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:24.644956 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5413c6-1195-4cc0-bd0d-d7b1cf739bbf admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:24.645380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5413c6-1195-4cc0-bd0d-d7b1cf739bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:24.646293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5413c6-1195-4cc0-bd0d-d7b1cf739bbf admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:24.646893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5413c6-1195-4cc0-bd0d-d7b1cf739bbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:24.649022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5413c6-1195-4cc0-bd0d-d7b1cf739bbf admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:24.675638 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1368/2711] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:52:24 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:52:24.679962 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f89b31-8fa6-4b34-a2a8-a8fb83cd9e0e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:24.692304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f89b31-8fa6-4b34-a2a8-a8fb83cd9e0e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:24.699416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f31329-4161-491f-b0f4-9df5ca7cacba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:24.700583 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f31329-4161-491f-b0f4-9df5ca7cacba admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:24.700862 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f31329-4161-491f-b0f4-9df5ca7cacba admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:24.701187 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f31329-4161-491f-b0f4-9df5ca7cacba admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:24.701897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f31329-4161-491f-b0f4-9df5ca7cacba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:24.703534 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f31329-4161-491f-b0f4-9df5ca7cacba admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:24.716765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1344/2712] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:24 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:52:24.721585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5413c6-1195-4cc0-bd0d-d7b1cf739bbf admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:24.733531 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5413c6-1195-4cc0-bd0d-d7b1cf739bbf admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:24.738202 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70415949-3b0b-4e23-83ef-17cd7b1800c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:24.738907 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70415949-3b0b-4e23-83ef-17cd7b1800c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:24.739181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70415949-3b0b-4e23-83ef-17cd7b1800c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:24.739466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70415949-3b0b-4e23-83ef-17cd7b1800c1 admin admin] PATH_INFO: `/v3/projects/c0ee3afe999044659147eed6567c92a8/users/61c38768a1bd46cdb6888f97ea3b2b19/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:24.760601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70415949-3b0b-4e23-83ef-17cd7b1800c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ee3afe999044659147eed6567c92a8, user_id=61c38768a1bd46cdb6888f97ea3b2b19, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:24.763344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70415949-3b0b-4e23-83ef-17cd7b1800c1 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:24.778237 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1369/2713] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:52:24 2022] PUT /identity/v3/projects/c0ee3afe999044659147eed6567c92a8/users/61c38768a1bd46cdb6888f97ea3b2b19/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:52:24.783080 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f31329-4161-491f-b0f4-9df5ca7cacba admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:24.794599 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f31329-4161-491f-b0f4-9df5ca7cacba admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:24.867233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71d49e-4e9c-4b17-8102-06a331e988c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:24.867860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71d49e-4e9c-4b17-8102-06a331e988c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:24.868145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71d49e-4e9c-4b17-8102-06a331e988c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:24.868422 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71d49e-4e9c-4b17-8102-06a331e988c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:24.869154 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71d49e-4e9c-4b17-8102-06a331e988c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:24.871659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71d49e-4e9c-4b17-8102-06a331e988c1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:24.882209 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1345/2714] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:24 2022] GET /identity/v3/roles => generated 1427 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:52:24.887155 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70415949-3b0b-4e23-83ef-17cd7b1800c1 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:24.900909 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70415949-3b0b-4e23-83ef-17cd7b1800c1 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:24.906259 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce70a7d-ddef-4720-a207-07244b601f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:24.906959 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce70a7d-ddef-4720-a207-07244b601f04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:24.907240 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce70a7d-ddef-4720-a207-07244b601f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:24.907519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce70a7d-ddef-4720-a207-07244b601f04 admin admin] PATH_INFO: `/v3/projects/c0ee3afe999044659147eed6567c92a8/users/61c38768a1bd46cdb6888f97ea3b2b19/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:24.909524 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce70a7d-ddef-4720-a207-07244b601f04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ee3afe999044659147eed6567c92a8, user_id=61c38768a1bd46cdb6888f97ea3b2b19, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:24.912172 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce70a7d-ddef-4720-a207-07244b601f04 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:24.929713 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1370/2715] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:52:24 2022] PUT /identity/v3/projects/c0ee3afe999044659147eed6567c92a8/users/61c38768a1bd46cdb6888f97ea3b2b19/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:24.941376 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84b492-e3e3-4659-b60b-0f01076de9b5 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:24.941664 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84b492-e3e3-4659-b60b-0f01076de9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:24.941977 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84b492-e3e3-4659-b60b-0f01076de9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:24.972714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1a84b492-e3e3-4659-b60b-0f01076de9b5 None None] MFA Rules not processed for user `61c38768a1bd46cdb6888f97ea3b2b19`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:24.973983 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1a84b492-e3e3-4659-b60b-0f01076de9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:25.048214 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1346/2716] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:52:25.061042 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce70a7d-ddef-4720-a207-07244b601f04 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:25.063224 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-5ce70a7d-ddef-4720-a207-07244b601f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:25.137503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce70a7d-ddef-4720-a207-07244b601f04 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:25.141256 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f220470-3205-4cea-9aaa-115c5908a499 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:25.141887 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f220470-3205-4cea-9aaa-115c5908a499 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:25.142229 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f220470-3205-4cea-9aaa-115c5908a499 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:25.142557 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f220470-3205-4cea-9aaa-115c5908a499 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:25.152832 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f220470-3205-4cea-9aaa-115c5908a499 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:25.154334 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f220470-3205-4cea-9aaa-115c5908a499 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:25.164506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1371/2717] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:52:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:25.254450 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84b492-e3e3-4659-b60b-0f01076de9b5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:25.265466 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84b492-e3e3-4659-b60b-0f01076de9b5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:25.269057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7931e-cdd6-4ee6-9353-a8019d48404e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:25.269738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7931e-cdd6-4ee6-9353-a8019d48404e service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:25.270041 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7931e-cdd6-4ee6-9353-a8019d48404e service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:25.270299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7931e-cdd6-4ee6-9353-a8019d48404e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:25.323501 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:25.337034 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7931e-cdd6-4ee6-9353-a8019d48404e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:25.339215 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7931e-cdd6-4ee6-9353-a8019d48404e service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:25.348924 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1347/2718] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:52:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:30.411591 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f220470-3205-4cea-9aaa-115c5908a499 service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:30.433286 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:30.481026 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f220470-3205-4cea-9aaa-115c5908a499 service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:30.485499 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d59db-7426-4234-b2d7-e530cb253625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:30.486145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d59db-7426-4234-b2d7-e530cb253625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:30.486432 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d59db-7426-4234-b2d7-e530cb253625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:30.486706 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d59db-7426-4234-b2d7-e530cb253625 admin admin] PATH_INFO: `/v3/users/c6258cdbae5e410db4a2ba90b9be9da8` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:30.487855 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d59db-7426-4234-b2d7-e530cb253625 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6258cdbae5e410db4a2ba90b9be9da8)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:30.490930 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d59db-7426-4234-b2d7-e530cb253625 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:30.542639 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-bb7d59db-7426-4234-b2d7-e530cb253625 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6258cdbae5e410db4a2ba90b9be9da8'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:30.543301 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-bb7d59db-7426-4234-b2d7-e530cb253625 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6258cdbae5e410db4a2ba90b9be9da8'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:30.552372 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-bb7d59db-7426-4234-b2d7-e530cb253625 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6258cdbae5e410db4a2ba90b9be9da8'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:30.566718 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-bb7d59db-7426-4234-b2d7-e530cb253625 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6258cdbae5e410db4a2ba90b9be9da8'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:30.575185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-bb7d59db-7426-4234-b2d7-e530cb253625 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6258cdbae5e410db4a2ba90b9be9da8'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:30.581357 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1372/2719] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:52:30 2022] DELETE /identity/v3/users/c6258cdbae5e410db4a2ba90b9be9da8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:30.586050 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b536e47-2f0b-46cd-ac17-618735594dee None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:30.586704 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b536e47-2f0b-46cd-ac17-618735594dee None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:30.586704 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b536e47-2f0b-46cd-ac17-618735594dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:30.608895 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-7b536e47-2f0b-46cd-ac17-618735594dee None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:30.610202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7b536e47-2f0b-46cd-ac17-618735594dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:30.680208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1348/2720] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:52:30.691396 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d59db-7426-4234-b2d7-e530cb253625 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:30.692826 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-bb7d59db-7426-4234-b2d7-e530cb253625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:30.769856 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d59db-7426-4234-b2d7-e530cb253625 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:30.773884 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570f38e-9ec6-4a5d-a89c-2a7eca4d439e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:30.774503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570f38e-9ec6-4a5d-a89c-2a7eca4d439e service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:30.774790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570f38e-9ec6-4a5d-a89c-2a7eca4d439e service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:30.775065 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570f38e-9ec6-4a5d-a89c-2a7eca4d439e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:30.786173 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570f38e-9ec6-4a5d-a89c-2a7eca4d439e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:30.788382 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570f38e-9ec6-4a5d-a89c-2a7eca4d439e service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:30.802851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1373/2721] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:52:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:31.114524 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b536e47-2f0b-46cd-ac17-618735594dee None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:31.116866 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7b536e47-2f0b-46cd-ac17-618735594dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:31.132335 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b536e47-2f0b-46cd-ac17-618735594dee None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:31.139943 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a528bfd-9bba-4606-a030-18840702acb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:31.140906 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a528bfd-9bba-4606-a030-18840702acb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:31.141322 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a528bfd-9bba-4606-a030-18840702acb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:31.141664 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a528bfd-9bba-4606-a030-18840702acb4 admin admin] PATH_INFO: `/v3/projects/a216e62e78e147ff8b5d6c37eeee30a5` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:31.142966 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a528bfd-9bba-4606-a030-18840702acb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a216e62e78e147ff8b5d6c37eeee30a5)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:31.145010 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a528bfd-9bba-4606-a030-18840702acb4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:31.198368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8a528bfd-9bba-4606-a030-18840702acb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a216e62e78e147ff8b5d6c37eeee30a5', 'request_id': 'req-8a528bfd-9bba-4606-a030-18840702acb4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:31.205128 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8a528bfd-9bba-4606-a030-18840702acb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a216e62e78e147ff8b5d6c37eeee30a5', 'request_id': 'req-8a528bfd-9bba-4606-a030-18840702acb4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:31.216072 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8a528bfd-9bba-4606-a030-18840702acb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a216e62e78e147ff8b5d6c37eeee30a5', 'request_id': 'req-8a528bfd-9bba-4606-a030-18840702acb4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:31.229153 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8a528bfd-9bba-4606-a030-18840702acb4 admin admin] The token cache is being invalidate because project a216e62e78e147ff8b5d6c37eeee30a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:52:31.229453 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8a528bfd-9bba-4606-a030-18840702acb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:31.230119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8a528bfd-9bba-4606-a030-18840702acb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:31.233582 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1349/2722] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:52:31 2022] DELETE /identity/v3/projects/a216e62e78e147ff8b5d6c37eeee30a5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:31.235785 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570f38e-9ec6-4a5d-a89c-2a7eca4d439e service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:31.246358 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570f38e-9ec6-4a5d-a89c-2a7eca4d439e service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:31.277252 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:31.317346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58645d-f8ee-4963-845a-be43ed9b3c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:31.317995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58645d-f8ee-4963-845a-be43ed9b3c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:31.318284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58645d-f8ee-4963-845a-be43ed9b3c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:31.318566 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58645d-f8ee-4963-845a-be43ed9b3c92 admin admin] PATH_INFO: `/v3/users/61c38768a1bd46cdb6888f97ea3b2b19` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:31.319928 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c58645d-f8ee-4963-845a-be43ed9b3c92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61c38768a1bd46cdb6888f97ea3b2b19)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:31.321894 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c58645d-f8ee-4963-845a-be43ed9b3c92 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:31.379081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-1c58645d-f8ee-4963-845a-be43ed9b3c92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61c38768a1bd46cdb6888f97ea3b2b19'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:31.379746 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-1c58645d-f8ee-4963-845a-be43ed9b3c92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61c38768a1bd46cdb6888f97ea3b2b19'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:31.388708 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-1c58645d-f8ee-4963-845a-be43ed9b3c92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61c38768a1bd46cdb6888f97ea3b2b19'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:31.401953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-1c58645d-f8ee-4963-845a-be43ed9b3c92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61c38768a1bd46cdb6888f97ea3b2b19'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:31.409487 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-1c58645d-f8ee-4963-845a-be43ed9b3c92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61c38768a1bd46cdb6888f97ea3b2b19'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:31.414945 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1374/2723] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:52:31 2022] DELETE /identity/v3/users/61c38768a1bd46cdb6888f97ea3b2b19 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:31.723106 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a528bfd-9bba-4606-a030-18840702acb4 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:31.724595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8a528bfd-9bba-4606-a030-18840702acb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:31.743591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a528bfd-9bba-4606-a030-18840702acb4 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:31.833682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c8d83-76cf-42fc-b9ea-f21430acd86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:31.834237 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c8d83-76cf-42fc-b9ea-f21430acd86e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:31.834522 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c8d83-76cf-42fc-b9ea-f21430acd86e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:31.834793 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c8d83-76cf-42fc-b9ea-f21430acd86e admin admin] PATH_INFO: `/v3/projects/c0ee3afe999044659147eed6567c92a8` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:31.835953 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6c8d83-76cf-42fc-b9ea-f21430acd86e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0ee3afe999044659147eed6567c92a8)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:31.838516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6c8d83-76cf-42fc-b9ea-f21430acd86e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:31.849299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:31.892355 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dd6c8d83-76cf-42fc-b9ea-f21430acd86e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0ee3afe999044659147eed6567c92a8', 'request_id': 'req-dd6c8d83-76cf-42fc-b9ea-f21430acd86e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:31.898631 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dd6c8d83-76cf-42fc-b9ea-f21430acd86e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0ee3afe999044659147eed6567c92a8', 'request_id': 'req-dd6c8d83-76cf-42fc-b9ea-f21430acd86e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:31.908133 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dd6c8d83-76cf-42fc-b9ea-f21430acd86e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0ee3afe999044659147eed6567c92a8', 'request_id': 'req-dd6c8d83-76cf-42fc-b9ea-f21430acd86e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:31.919569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dd6c8d83-76cf-42fc-b9ea-f21430acd86e admin admin] The token cache is being invalidate because project c0ee3afe999044659147eed6567c92a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:52:31.919872 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dd6c8d83-76cf-42fc-b9ea-f21430acd86e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:31.920560 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dd6c8d83-76cf-42fc-b9ea-f21430acd86e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:31.922150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1350/2724] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:52:31 2022] DELETE /identity/v3/projects/c0ee3afe999044659147eed6567c92a8 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:31.929954 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e92d1-e4f2-47ac-9cbd-f08b421239c1 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:31.930400 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e92d1-e4f2-47ac-9cbd-f08b421239c1 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:31.930943 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e92d1-e4f2-47ac-9cbd-f08b421239c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:31.953066 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-e26e92d1-e4f2-47ac-9cbd-f08b421239c1 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:31.954853 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e26e92d1-e4f2-47ac-9cbd-f08b421239c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:32.022574 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1375/2725] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:32.031963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbad0b9-0387-4e34-8671-30bf1750ada5 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:32.032518 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbad0b9-0387-4e34-8671-30bf1750ada5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:32.032796 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbad0b9-0387-4e34-8671-30bf1750ada5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:32.053897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-cdbad0b9-0387-4e34-8671-30bf1750ada5 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:32.055101 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-cdbad0b9-0387-4e34-8671-30bf1750ada5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:32.065248 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1351/2726] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:52:32.070041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e92d1-e4f2-47ac-9cbd-f08b421239c1 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:32.080866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e92d1-e4f2-47ac-9cbd-f08b421239c1 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:32.085713 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b218e-732f-474f-bae8-892b9a24f07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:32.086270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b218e-732f-474f-bae8-892b9a24f07e admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:32.086556 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b218e-732f-474f-bae8-892b9a24f07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:32.086833 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b218e-732f-474f-bae8-892b9a24f07e admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:32.087584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b218e-732f-474f-bae8-892b9a24f07e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:52:32.087900 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b218e-732f-474f-bae8-892b9a24f07e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:32.089632 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b218e-732f-474f-bae8-892b9a24f07e admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:32.101711 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1376/2727] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:52:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:52:32.106449 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbad0b9-0387-4e34-8671-30bf1750ada5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:32.116045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbad0b9-0387-4e34-8671-30bf1750ada5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:32.121581 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9f8b3-df42-4ade-b00d-52afe7c2785d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:32.122206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9f8b3-df42-4ade-b00d-52afe7c2785d admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:32.122484 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9f8b3-df42-4ade-b00d-52afe7c2785d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:32.122758 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9f8b3-df42-4ade-b00d-52afe7c2785d admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:32.123443 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9f8b3-df42-4ade-b00d-52afe7c2785d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:32.125541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9f8b3-df42-4ade-b00d-52afe7c2785d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:32.158311 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1352/2728] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:52:32 2022] POST /identity/v3/projects => generated 388 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:52:32.159786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b218e-732f-474f-bae8-892b9a24f07e admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:32.170133 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b218e-732f-474f-bae8-892b9a24f07e admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:32.211962 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:32.247127 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa391d5-d9d4-451c-a6b7-998c2d53dd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:32.247760 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa391d5-d9d4-451c-a6b7-998c2d53dd6f admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:32.248040 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa391d5-d9d4-451c-a6b7-998c2d53dd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:32.248320 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa391d5-d9d4-451c-a6b7-998c2d53dd6f admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:32.248985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa391d5-d9d4-451c-a6b7-998c2d53dd6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:32.250946 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa391d5-d9d4-451c-a6b7-998c2d53dd6f admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:32.273798 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1377/2729] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:52:32 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:52:32.278258 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9f8b3-df42-4ade-b00d-52afe7c2785d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:32.288854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9f8b3-df42-4ade-b00d-52afe7c2785d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:32.293674 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8391c-27f9-42ba-a416-607329621b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:32.294294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8391c-27f9-42ba-a416-607329621b37 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:32.294575 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8391c-27f9-42ba-a416-607329621b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:32.294847 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8391c-27f9-42ba-a416-607329621b37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:32.295551 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c8391c-27f9-42ba-a416-607329621b37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:32.297191 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c8391c-27f9-42ba-a416-607329621b37 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:32.308774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1353/2730] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:32 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:52:32.313505 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa391d5-d9d4-451c-a6b7-998c2d53dd6f admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:32.324977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa391d5-d9d4-451c-a6b7-998c2d53dd6f admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:32.329946 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5925559-b8b2-4e7b-87a0-c48c7f926ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:32.330731 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5925559-b8b2-4e7b-87a0-c48c7f926ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:32.331057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5925559-b8b2-4e7b-87a0-c48c7f926ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:32.331474 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5925559-b8b2-4e7b-87a0-c48c7f926ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:32.332173 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5925559-b8b2-4e7b-87a0-c48c7f926ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:32.333832 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5925559-b8b2-4e7b-87a0-c48c7f926ed2 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:32.345784 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1378/2731] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:32 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:52:32.351143 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8391c-27f9-42ba-a416-607329621b37 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:32.364582 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8391c-27f9-42ba-a416-607329621b37 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:32.370008 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3effd-94b7-4df1-a8ba-88a75c69549d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:32.370786 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3effd-94b7-4df1-a8ba-88a75c69549d admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:32.371119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3effd-94b7-4df1-a8ba-88a75c69549d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:32.371446 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3effd-94b7-4df1-a8ba-88a75c69549d admin admin] PATH_INFO: `/v3/projects/8344a77715724deea4f5e6fa4eaf75e9/users/d24fa7e62db04ab5b1281249bb387538/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:32.387298 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb3effd-94b7-4df1-a8ba-88a75c69549d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8344a77715724deea4f5e6fa4eaf75e9, user_id=d24fa7e62db04ab5b1281249bb387538, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:32.389931 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb3effd-94b7-4df1-a8ba-88a75c69549d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:32.404570 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1354/2732] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:52:32 2022] PUT /identity/v3/projects/8344a77715724deea4f5e6fa4eaf75e9/users/d24fa7e62db04ab5b1281249bb387538/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:32.414950 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c747e93-3e04-4e36-b25e-025e2cc8a467 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:32.416215 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c747e93-3e04-4e36-b25e-025e2cc8a467 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:32.416215 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c747e93-3e04-4e36-b25e-025e2cc8a467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:32.445275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-8c747e93-3e04-4e36-b25e-025e2cc8a467 None None] MFA Rules not processed for user `d24fa7e62db04ab5b1281249bb387538`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:32.446435 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-8c747e93-3e04-4e36-b25e-025e2cc8a467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:32.524090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1379/2733] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:32 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:52:32.530393 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3effd-94b7-4df1-a8ba-88a75c69549d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:32.599996 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3effd-94b7-4df1-a8ba-88a75c69549d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:32.605435 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535b696-eb89-4f6c-bcfa-227d4b8e1e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:32.606052 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535b696-eb89-4f6c-bcfa-227d4b8e1e19 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:32.606334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535b696-eb89-4f6c-bcfa-227d4b8e1e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:32.606617 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535b696-eb89-4f6c-bcfa-227d4b8e1e19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:32.607294 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535b696-eb89-4f6c-bcfa-227d4b8e1e19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:32.609204 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535b696-eb89-4f6c-bcfa-227d4b8e1e19 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:32.639032 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1355/2734] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:52:32 2022] POST /identity/v3/projects => generated 386 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:52:32.643301 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c747e93-3e04-4e36-b25e-025e2cc8a467 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:32.654881 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c747e93-3e04-4e36-b25e-025e2cc8a467 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:32.724102 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1eebfe-0697-49d6-9200-bb5d94f7ab5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:32.724716 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1eebfe-0697-49d6-9200-bb5d94f7ab5a admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:32.724997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1eebfe-0697-49d6-9200-bb5d94f7ab5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:32.725310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1eebfe-0697-49d6-9200-bb5d94f7ab5a admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:32.725972 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1eebfe-0697-49d6-9200-bb5d94f7ab5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:32.728020 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1eebfe-0697-49d6-9200-bb5d94f7ab5a admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:32.754436 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1380/2735] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:52:32 2022] POST /identity/v3/users => generated 300 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:52:32.759005 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535b696-eb89-4f6c-bcfa-227d4b8e1e19 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:32.770666 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535b696-eb89-4f6c-bcfa-227d4b8e1e19 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:32.775963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e5af6-11f8-4b51-b89b-82740f3e99fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:32.776579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e5af6-11f8-4b51-b89b-82740f3e99fe admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:32.776852 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e5af6-11f8-4b51-b89b-82740f3e99fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:32.777143 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e5af6-11f8-4b51-b89b-82740f3e99fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:32.777831 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e5af6-11f8-4b51-b89b-82740f3e99fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:32.779418 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e5af6-11f8-4b51-b89b-82740f3e99fe admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:32.790485 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1356/2736] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:32 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:52:32.795925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1eebfe-0697-49d6-9200-bb5d94f7ab5a admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:32.807557 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1eebfe-0697-49d6-9200-bb5d94f7ab5a admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:32.812671 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294359c-25be-4085-aa27-23ada9ee4251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:32.813412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294359c-25be-4085-aa27-23ada9ee4251 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:32.813725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294359c-25be-4085-aa27-23ada9ee4251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:32.814008 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294359c-25be-4085-aa27-23ada9ee4251 admin admin] PATH_INFO: `/v3/projects/ae40315a1f974687bac363ad870acd37/users/7a16f3bf7f254b1381cdc2fc8414d07e/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:32.831093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294359c-25be-4085-aa27-23ada9ee4251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae40315a1f974687bac363ad870acd37, user_id=7a16f3bf7f254b1381cdc2fc8414d07e, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:32.833858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294359c-25be-4085-aa27-23ada9ee4251 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:32.849991 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1381/2737] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:52:32 2022] PUT /identity/v3/projects/ae40315a1f974687bac363ad870acd37/users/7a16f3bf7f254b1381cdc2fc8414d07e/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:52:32.855511 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e5af6-11f8-4b51-b89b-82740f3e99fe admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:32.865912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e5af6-11f8-4b51-b89b-82740f3e99fe admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:32.871176 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:32.939803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8fe29-7340-4979-a24e-344de8624815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:32.940464 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8fe29-7340-4979-a24e-344de8624815 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:32.940745 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8fe29-7340-4979-a24e-344de8624815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:32.941018 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8fe29-7340-4979-a24e-344de8624815 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:32.941802 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8fe29-7340-4979-a24e-344de8624815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:32.943383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8fe29-7340-4979-a24e-344de8624815 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:32.953696 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1357/2738] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:32 2022] GET /identity/v3/roles => generated 1427 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:52:32.958063 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294359c-25be-4085-aa27-23ada9ee4251 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:32.969766 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294359c-25be-4085-aa27-23ada9ee4251 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:32.974683 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac5441-721a-45e5-9765-57a197296d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:32.975490 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac5441-721a-45e5-9765-57a197296d3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:32.975776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac5441-721a-45e5-9765-57a197296d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:32.976055 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac5441-721a-45e5-9765-57a197296d3c admin admin] PATH_INFO: `/v3/projects/ae40315a1f974687bac363ad870acd37/users/7a16f3bf7f254b1381cdc2fc8414d07e/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:32.978139 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ac5441-721a-45e5-9765-57a197296d3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae40315a1f974687bac363ad870acd37, user_id=7a16f3bf7f254b1381cdc2fc8414d07e, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:32.981466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ac5441-721a-45e5-9765-57a197296d3c admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:32.987537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:32.994276 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1382/2739] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:52:32 2022] PUT /identity/v3/projects/ae40315a1f974687bac363ad870acd37/users/7a16f3bf7f254b1381cdc2fc8414d07e/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:33.005440 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dac0ef-25ad-45b6-b681-eaf1d6458c78 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:33.005742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dac0ef-25ad-45b6-b681-eaf1d6458c78 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:33.006035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dac0ef-25ad-45b6-b681-eaf1d6458c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:33.039566 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-21dac0ef-25ad-45b6-b681-eaf1d6458c78 None None] MFA Rules not processed for user `7a16f3bf7f254b1381cdc2fc8414d07e`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:33.041421 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-21dac0ef-25ad-45b6-b681-eaf1d6458c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:33.133818 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1358/2740] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:33 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:52:33.147304 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac5441-721a-45e5-9765-57a197296d3c admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:33.149521 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-94ac5441-721a-45e5-9765-57a197296d3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:33.223989 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac5441-721a-45e5-9765-57a197296d3c admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:33.227764 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a87755-b238-40e6-8d4e-6c5dbab2b6ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:33.228460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a87755-b238-40e6-8d4e-6c5dbab2b6ba service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:33.228741 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a87755-b238-40e6-8d4e-6c5dbab2b6ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:33.229021 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a87755-b238-40e6-8d4e-6c5dbab2b6ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:33.238359 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a87755-b238-40e6-8d4e-6c5dbab2b6ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:33.239814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a87755-b238-40e6-8d4e-6c5dbab2b6ba service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:33.251021 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1383/2741] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:52:33 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:33.891117 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dac0ef-25ad-45b6-b681-eaf1d6458c78 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:33.963335 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dac0ef-25ad-45b6-b681-eaf1d6458c78 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:33.968970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a30f8-572b-4b83-9e82-9325c0c02764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:33.969811 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a30f8-572b-4b83-9e82-9325c0c02764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:33.970099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a30f8-572b-4b83-9e82-9325c0c02764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:33.970374 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a30f8-572b-4b83-9e82-9325c0c02764 admin admin] PATH_INFO: `/v3/users/d24fa7e62db04ab5b1281249bb387538` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:33.971581 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a30f8-572b-4b83-9e82-9325c0c02764 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d24fa7e62db04ab5b1281249bb387538)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:33.973548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a30f8-572b-4b83-9e82-9325c0c02764 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:34.015372 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:34.031203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b95a30f8-572b-4b83-9e82-9325c0c02764 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd24fa7e62db04ab5b1281249bb387538'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:34.038322 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b95a30f8-572b-4b83-9e82-9325c0c02764 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd24fa7e62db04ab5b1281249bb387538'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:34.039135 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b95a30f8-572b-4b83-9e82-9325c0c02764 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd24fa7e62db04ab5b1281249bb387538'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:34.048044 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b95a30f8-572b-4b83-9e82-9325c0c02764 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd24fa7e62db04ab5b1281249bb387538'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:34.048817 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b95a30f8-572b-4b83-9e82-9325c0c02764 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd24fa7e62db04ab5b1281249bb387538'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:34.068016 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1359/2742] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:52:33 2022] DELETE /identity/v3/users/d24fa7e62db04ab5b1281249bb387538 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:34.073768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382410ca-2a1c-481d-a9d5-fc448f3fba4a None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:34.074430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382410ca-2a1c-481d-a9d5-fc448f3fba4a None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:34.074430 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382410ca-2a1c-481d-a9d5-fc448f3fba4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:34.094849 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-382410ca-2a1c-481d-a9d5-fc448f3fba4a None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:34.096193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-382410ca-2a1c-481d-a9d5-fc448f3fba4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:34.185342 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1384/2743] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:34.196449 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a30f8-572b-4b83-9e82-9325c0c02764 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:34.198616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b95a30f8-572b-4b83-9e82-9325c0c02764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:34.262580 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a30f8-572b-4b83-9e82-9325c0c02764 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:34.266447 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921f159-a938-4953-960f-7f9133ea200d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:34.267077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921f159-a938-4953-960f-7f9133ea200d service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:34.267362 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921f159-a938-4953-960f-7f9133ea200d service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:34.267641 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921f159-a938-4953-960f-7f9133ea200d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:34.277488 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921f159-a938-4953-960f-7f9133ea200d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:34.278922 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921f159-a938-4953-960f-7f9133ea200d service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:34.288829 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1360/2744] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:52:34 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:34.577923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382410ca-2a1c-481d-a9d5-fc448f3fba4a None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:34.579406 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-382410ca-2a1c-481d-a9d5-fc448f3fba4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:34.591933 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382410ca-2a1c-481d-a9d5-fc448f3fba4a None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:34.599426 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75145a7e-35ae-4344-8ffe-113430a12bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:34.600625 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75145a7e-35ae-4344-8ffe-113430a12bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:34.601048 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75145a7e-35ae-4344-8ffe-113430a12bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:34.601492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75145a7e-35ae-4344-8ffe-113430a12bf6 admin admin] PATH_INFO: `/v3/projects/8344a77715724deea4f5e6fa4eaf75e9` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:34.603679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75145a7e-35ae-4344-8ffe-113430a12bf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8344a77715724deea4f5e6fa4eaf75e9)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:34.605766 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75145a7e-35ae-4344-8ffe-113430a12bf6 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:34.673742 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-75145a7e-35ae-4344-8ffe-113430a12bf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8344a77715724deea4f5e6fa4eaf75e9', 'request_id': 'req-75145a7e-35ae-4344-8ffe-113430a12bf6'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:34.683745 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-75145a7e-35ae-4344-8ffe-113430a12bf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8344a77715724deea4f5e6fa4eaf75e9', 'request_id': 'req-75145a7e-35ae-4344-8ffe-113430a12bf6'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:34.693806 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-75145a7e-35ae-4344-8ffe-113430a12bf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8344a77715724deea4f5e6fa4eaf75e9', 'request_id': 'req-75145a7e-35ae-4344-8ffe-113430a12bf6'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:34.705271 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-75145a7e-35ae-4344-8ffe-113430a12bf6 admin admin] The token cache is being invalidate because project 8344a77715724deea4f5e6fa4eaf75e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:52:34.705567 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-75145a7e-35ae-4344-8ffe-113430a12bf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:34.706283 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-75145a7e-35ae-4344-8ffe-113430a12bf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:34.707831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1385/2745] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:52:34 2022] DELETE /identity/v3/projects/8344a77715724deea4f5e6fa4eaf75e9 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:34.712107 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921f159-a938-4953-960f-7f9133ea200d service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:34.724134 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921f159-a938-4953-960f-7f9133ea200d service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:34.798674 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47b1a5-0a0c-4513-8cee-555793f1a2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:34.799334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47b1a5-0a0c-4513-8cee-555793f1a2ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:34.800741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47b1a5-0a0c-4513-8cee-555793f1a2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:34.800741 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47b1a5-0a0c-4513-8cee-555793f1a2ea admin admin] PATH_INFO: `/v3/users/7a16f3bf7f254b1381cdc2fc8414d07e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:34.801032 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47b1a5-0a0c-4513-8cee-555793f1a2ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a16f3bf7f254b1381cdc2fc8414d07e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:34.803111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47b1a5-0a0c-4513-8cee-555793f1a2ea admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:34.860422 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9e47b1a5-0a0c-4513-8cee-555793f1a2ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a16f3bf7f254b1381cdc2fc8414d07e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:34.869635 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:34.872819 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9e47b1a5-0a0c-4513-8cee-555793f1a2ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a16f3bf7f254b1381cdc2fc8414d07e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:34.874174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9e47b1a5-0a0c-4513-8cee-555793f1a2ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a16f3bf7f254b1381cdc2fc8414d07e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:34.886510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9e47b1a5-0a0c-4513-8cee-555793f1a2ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a16f3bf7f254b1381cdc2fc8414d07e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:34.887086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9e47b1a5-0a0c-4513-8cee-555793f1a2ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a16f3bf7f254b1381cdc2fc8414d07e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:34.913821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1361/2746] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:52:34 2022] DELETE /identity/v3/users/7a16f3bf7f254b1381cdc2fc8414d07e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:35.229249 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75145a7e-35ae-4344-8ffe-113430a12bf6 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:35.231393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-75145a7e-35ae-4344-8ffe-113430a12bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:35.246735 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75145a7e-35ae-4344-8ffe-113430a12bf6 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:35.274229 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:35.316112 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c36a32-0062-4c3c-a155-5849b514e944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:35.316764 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c36a32-0062-4c3c-a155-5849b514e944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:35.317041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c36a32-0062-4c3c-a155-5849b514e944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:35.317345 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c36a32-0062-4c3c-a155-5849b514e944 admin admin] PATH_INFO: `/v3/projects/ae40315a1f974687bac363ad870acd37` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:35.318531 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c36a32-0062-4c3c-a155-5849b514e944 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae40315a1f974687bac363ad870acd37)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:35.320630 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c36a32-0062-4c3c-a155-5849b514e944 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:35.372198 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-68c36a32-0062-4c3c-a155-5849b514e944 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae40315a1f974687bac363ad870acd37', 'request_id': 'req-68c36a32-0062-4c3c-a155-5849b514e944'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:35.380143 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-68c36a32-0062-4c3c-a155-5849b514e944 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae40315a1f974687bac363ad870acd37', 'request_id': 'req-68c36a32-0062-4c3c-a155-5849b514e944'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:35.391749 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-68c36a32-0062-4c3c-a155-5849b514e944 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae40315a1f974687bac363ad870acd37', 'request_id': 'req-68c36a32-0062-4c3c-a155-5849b514e944'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:35.403658 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-68c36a32-0062-4c3c-a155-5849b514e944 admin admin] The token cache is being invalidate because project ae40315a1f974687bac363ad870acd37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:52:35.403978 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-68c36a32-0062-4c3c-a155-5849b514e944 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:35.404645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-68c36a32-0062-4c3c-a155-5849b514e944 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:35.406254 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1386/2747] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:52:35 2022] DELETE /identity/v3/projects/ae40315a1f974687bac363ad870acd37 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:35.411372 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa07bfd-387c-451c-aac0-b30e3bc62617 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:35.411734 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa07bfd-387c-451c-aac0-b30e3bc62617 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:35.412064 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa07bfd-387c-451c-aac0-b30e3bc62617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:35.434739 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5fa07bfd-387c-451c-aac0-b30e3bc62617 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:35.435949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5fa07bfd-387c-451c-aac0-b30e3bc62617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:35.504972 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1362/2748] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:35.514145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d765d6-fa84-47cf-9c67-5476931744b1 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:35.514567 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d765d6-fa84-47cf-9c67-5476931744b1 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:35.515007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d765d6-fa84-47cf-9c67-5476931744b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:35.536492 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-c9d765d6-fa84-47cf-9c67-5476931744b1 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:35.537733 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c9d765d6-fa84-47cf-9c67-5476931744b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:35.546133 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1387/2749] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:35.550478 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa07bfd-387c-451c-aac0-b30e3bc62617 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:35.561493 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa07bfd-387c-451c-aac0-b30e3bc62617 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:35.566410 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339f234-c56e-4abb-aec9-eb248ab3a21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:35.567043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339f234-c56e-4abb-aec9-eb248ab3a21b admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:35.567489 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339f234-c56e-4abb-aec9-eb248ab3a21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:35.567768 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339f234-c56e-4abb-aec9-eb248ab3a21b admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:35.568486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339f234-c56e-4abb-aec9-eb248ab3a21b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:52:35.568842 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339f234-c56e-4abb-aec9-eb248ab3a21b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:35.570591 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339f234-c56e-4abb-aec9-eb248ab3a21b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:35.583740 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1363/2750] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:52:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:52:35.591995 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d765d6-fa84-47cf-9c67-5476931744b1 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:35.609342 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d765d6-fa84-47cf-9c67-5476931744b1 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:35.617061 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08785c3a-eb86-4617-9158-9ea96f8f7dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:35.618145 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08785c3a-eb86-4617-9158-9ea96f8f7dca admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:35.618679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08785c3a-eb86-4617-9158-9ea96f8f7dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:35.619163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08785c3a-eb86-4617-9158-9ea96f8f7dca admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:35.620353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08785c3a-eb86-4617-9158-9ea96f8f7dca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:35.624001 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08785c3a-eb86-4617-9158-9ea96f8f7dca admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:35.665925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1388/2751] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:52:35 2022] POST /identity/v3/projects => generated 386 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:52:35.671805 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339f234-c56e-4abb-aec9-eb248ab3a21b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:35.683434 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339f234-c56e-4abb-aec9-eb248ab3a21b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:35.747925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fc42e-674c-4e74-884e-b3f639396aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:35.748548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fc42e-674c-4e74-884e-b3f639396aae admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:35.748838 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fc42e-674c-4e74-884e-b3f639396aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:35.749147 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fc42e-674c-4e74-884e-b3f639396aae admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:35.749805 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2fc42e-674c-4e74-884e-b3f639396aae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:35.751834 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2fc42e-674c-4e74-884e-b3f639396aae admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:35.777177 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1364/2752] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:52:35 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:52:35.781481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08785c3a-eb86-4617-9158-9ea96f8f7dca admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:35.793263 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08785c3a-eb86-4617-9158-9ea96f8f7dca admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:35.797981 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ae0d2-baa9-4941-b377-83ef561227c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:35.798592 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ae0d2-baa9-4941-b377-83ef561227c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:35.798880 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ae0d2-baa9-4941-b377-83ef561227c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:35.799161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ae0d2-baa9-4941-b377-83ef561227c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:35.799867 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ae0d2-baa9-4941-b377-83ef561227c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:35.801517 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ae0d2-baa9-4941-b377-83ef561227c9 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:35.814273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1389/2753] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:35 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:52:35.818675 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fc42e-674c-4e74-884e-b3f639396aae admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:35.829852 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fc42e-674c-4e74-884e-b3f639396aae admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:35.834525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64116b7a-9430-4c79-8e9e-71dff1bdd1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:35.835142 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64116b7a-9430-4c79-8e9e-71dff1bdd1da admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:35.835430 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64116b7a-9430-4c79-8e9e-71dff1bdd1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:35.835738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64116b7a-9430-4c79-8e9e-71dff1bdd1da admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:35.836427 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64116b7a-9430-4c79-8e9e-71dff1bdd1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:35.838114 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64116b7a-9430-4c79-8e9e-71dff1bdd1da admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:35.849860 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1365/2754] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:35 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:52:35.854792 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ae0d2-baa9-4941-b377-83ef561227c9 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:35.866161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ae0d2-baa9-4941-b377-83ef561227c9 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:35.872786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcdd7b-15f3-4595-b3cd-9a5f05053aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:35.873529 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcdd7b-15f3-4595-b3cd-9a5f05053aee admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:35.873819 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcdd7b-15f3-4595-b3cd-9a5f05053aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:35.874100 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcdd7b-15f3-4595-b3cd-9a5f05053aee admin admin] PATH_INFO: `/v3/projects/fcb3ca74899a47cd80a28f3f82292e77/users/fd6686dcd1f24bf3bff7481acb32b25b/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:35.895366 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fcdd7b-15f3-4595-b3cd-9a5f05053aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcb3ca74899a47cd80a28f3f82292e77, user_id=fd6686dcd1f24bf3bff7481acb32b25b, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:35.898076 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fcdd7b-15f3-4595-b3cd-9a5f05053aee admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:35.912670 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1390/2755] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:52:35 2022] PUT /identity/v3/projects/fcb3ca74899a47cd80a28f3f82292e77/users/fd6686dcd1f24bf3bff7481acb32b25b/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:52:35.932232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0daaf63-f5ce-45d7-ac70-a23e72faabf5 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:35.932888 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0daaf63-f5ce-45d7-ac70-a23e72faabf5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:35.933238 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0daaf63-f5ce-45d7-ac70-a23e72faabf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:35.972682 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d0daaf63-f5ce-45d7-ac70-a23e72faabf5 None None] MFA Rules not processed for user `fd6686dcd1f24bf3bff7481acb32b25b`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:35.973962 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d0daaf63-f5ce-45d7-ac70-a23e72faabf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:36.006996 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:36.052088 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1366/2756] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:35 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:36.064287 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcdd7b-15f3-4595-b3cd-9a5f05053aee admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:36.066058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-98fcdd7b-15f3-4595-b3cd-9a5f05053aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:36.152440 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcdd7b-15f3-4595-b3cd-9a5f05053aee admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:36.156191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c1a3c-8224-494d-ac29-46b720f69332 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:36.156814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c1a3c-8224-494d-ac29-46b720f69332 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:36.157090 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c1a3c-8224-494d-ac29-46b720f69332 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:36.157401 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c1a3c-8224-494d-ac29-46b720f69332 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:36.162934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:36.169299 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c1a3c-8224-494d-ac29-46b720f69332 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:36.170957 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c1a3c-8224-494d-ac29-46b720f69332 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:36.183167 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1391/2757] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:52:36 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:44.864771 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0daaf63-f5ce-45d7-ac70-a23e72faabf5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:44.950077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0daaf63-f5ce-45d7-ac70-a23e72faabf5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:44.954852 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16ce75-3c54-4aab-b10d-532d5bc98f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:44.955491 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16ce75-3c54-4aab-b10d-532d5bc98f87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:44.955776 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16ce75-3c54-4aab-b10d-532d5bc98f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:44.956047 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16ce75-3c54-4aab-b10d-532d5bc98f87 admin admin] PATH_INFO: `/v3/users/fd6686dcd1f24bf3bff7481acb32b25b` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:44.957314 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16ce75-3c54-4aab-b10d-532d5bc98f87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd6686dcd1f24bf3bff7481acb32b25b)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:44.959303 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16ce75-3c54-4aab-b10d-532d5bc98f87 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:45.012263 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1a16ce75-3c54-4aab-b10d-532d5bc98f87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd6686dcd1f24bf3bff7481acb32b25b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:45.020956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1a16ce75-3c54-4aab-b10d-532d5bc98f87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd6686dcd1f24bf3bff7481acb32b25b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:45.021704 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1a16ce75-3c54-4aab-b10d-532d5bc98f87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd6686dcd1f24bf3bff7481acb32b25b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:45.032233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1a16ce75-3c54-4aab-b10d-532d5bc98f87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd6686dcd1f24bf3bff7481acb32b25b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:45.033090 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1a16ce75-3c54-4aab-b10d-532d5bc98f87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd6686dcd1f24bf3bff7481acb32b25b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:45.054713 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1367/2758] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:52:44 2022] DELETE /identity/v3/users/fd6686dcd1f24bf3bff7481acb32b25b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:45.063194 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b53a64-2385-45fe-8207-9d95d1d4c6c2 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:45.063918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b53a64-2385-45fe-8207-9d95d1d4c6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:45.064106 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b53a64-2385-45fe-8207-9d95d1d4c6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:45.090598 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-a5b53a64-2385-45fe-8207-9d95d1d4c6c2 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:45.092491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-a5b53a64-2385-45fe-8207-9d95d1d4c6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:45.162326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1392/2759] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:45.177888 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16ce75-3c54-4aab-b10d-532d5bc98f87 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:45.180177 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1a16ce75-3c54-4aab-b10d-532d5bc98f87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:45.247939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:45.256421 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16ce75-3c54-4aab-b10d-532d5bc98f87 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:45.263981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1fd604-d528-431a-8671-f1d59785d0ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:45.264687 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fd604-d528-431a-8671-f1d59785d0ce service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:45.265286 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fd604-d528-431a-8671-f1d59785d0ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:45.265571 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fd604-d528-431a-8671-f1d59785d0ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:45.279143 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1fd604-d528-431a-8671-f1d59785d0ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:45.280600 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1fd604-d528-431a-8671-f1d59785d0ce service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:45.290952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1368/2760] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:52:45 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:45.594236 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b53a64-2385-45fe-8207-9d95d1d4c6c2 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:45.595833 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-a5b53a64-2385-45fe-8207-9d95d1d4c6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:45.612048 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b53a64-2385-45fe-8207-9d95d1d4c6c2 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:45.620727 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9767694-d54c-4df1-82a6-438284c59bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:45.621439 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9767694-d54c-4df1-82a6-438284c59bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:45.621724 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9767694-d54c-4df1-82a6-438284c59bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:45.622005 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9767694-d54c-4df1-82a6-438284c59bdf admin admin] PATH_INFO: `/v3/projects/fcb3ca74899a47cd80a28f3f82292e77` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:45.623219 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9767694-d54c-4df1-82a6-438284c59bdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcb3ca74899a47cd80a28f3f82292e77)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:45.625321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9767694-d54c-4df1-82a6-438284c59bdf admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:45.682692 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-a9767694-d54c-4df1-82a6-438284c59bdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcb3ca74899a47cd80a28f3f82292e77', 'request_id': 'req-a9767694-d54c-4df1-82a6-438284c59bdf'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:45.689346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-a9767694-d54c-4df1-82a6-438284c59bdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcb3ca74899a47cd80a28f3f82292e77', 'request_id': 'req-a9767694-d54c-4df1-82a6-438284c59bdf'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:45.698199 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-a9767694-d54c-4df1-82a6-438284c59bdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcb3ca74899a47cd80a28f3f82292e77', 'request_id': 'req-a9767694-d54c-4df1-82a6-438284c59bdf'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:45.709133 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-a9767694-d54c-4df1-82a6-438284c59bdf admin admin] The token cache is being invalidate because project fcb3ca74899a47cd80a28f3f82292e77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:52:45.709447 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-a9767694-d54c-4df1-82a6-438284c59bdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:45.710106 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-a9767694-d54c-4df1-82a6-438284c59bdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:52:45.711903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1393/2761] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:52:45 2022] DELETE /identity/v3/projects/fcb3ca74899a47cd80a28f3f82292e77 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:45.719316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1fc02-47df-428b-af6b-14ac76cbeba0 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:45.719765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1fc02-47df-428b-af6b-14ac76cbeba0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:45.720099 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1fc02-47df-428b-af6b-14ac76cbeba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:45.743887 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-fee1fc02-47df-428b-af6b-14ac76cbeba0 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:45.745348 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fee1fc02-47df-428b-af6b-14ac76cbeba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:45.810751 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1369/2762] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:45.819919 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721ecc9-fc9a-4bc4-a27e-b768c8c741af None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:45.820263 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721ecc9-fc9a-4bc4-a27e-b768c8c741af None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:45.820592 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721ecc9-fc9a-4bc4-a27e-b768c8c741af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:45.846541 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-c721ecc9-fc9a-4bc4-a27e-b768c8c741af None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:45.847821 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c721ecc9-fc9a-4bc4-a27e-b768c8c741af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:45.858057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1394/2763] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:45.862729 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1fc02-47df-428b-af6b-14ac76cbeba0 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:45.874602 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1fc02-47df-428b-af6b-14ac76cbeba0 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:45.879756 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e313fdb-7c91-476c-8735-0361c08ec415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:45.880381 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e313fdb-7c91-476c-8735-0361c08ec415 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:45.880709 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e313fdb-7c91-476c-8735-0361c08ec415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:45.880984 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e313fdb-7c91-476c-8735-0361c08ec415 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:45.881784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e313fdb-7c91-476c-8735-0361c08ec415 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:52:45.882098 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e313fdb-7c91-476c-8735-0361c08ec415 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:45.883821 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e313fdb-7c91-476c-8735-0361c08ec415 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:45.897245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1370/2764] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:52:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:52:45.902024 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721ecc9-fc9a-4bc4-a27e-b768c8c741af None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:45.912846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721ecc9-fc9a-4bc4-a27e-b768c8c741af None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:45.919310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b83f8-1523-49e6-957e-3c34f2ceff1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:45.920086 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b83f8-1523-49e6-957e-3c34f2ceff1e admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:45.920306 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b83f8-1523-49e6-957e-3c34f2ceff1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:45.920579 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b83f8-1523-49e6-957e-3c34f2ceff1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:45.921385 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b83f8-1523-49e6-957e-3c34f2ceff1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:45.923322 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b83f8-1523-49e6-957e-3c34f2ceff1e admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:45.955683 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1395/2765] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:52:45 2022] POST /identity/v3/projects => generated 382 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:52:45.960593 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e313fdb-7c91-476c-8735-0361c08ec415 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:45.971747 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e313fdb-7c91-476c-8735-0361c08ec415 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:46.036988 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a862b6-8d53-43b2-962f-1a2936083365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:46.037714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a862b6-8d53-43b2-962f-1a2936083365 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:46.038048 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a862b6-8d53-43b2-962f-1a2936083365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:46.038367 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a862b6-8d53-43b2-962f-1a2936083365 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:46.039188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a862b6-8d53-43b2-962f-1a2936083365 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:46.041678 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a862b6-8d53-43b2-962f-1a2936083365 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:46.065253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1371/2766] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:52:45 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:52:46.069752 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b83f8-1523-49e6-957e-3c34f2ceff1e admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:46.081641 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b83f8-1523-49e6-957e-3c34f2ceff1e admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:46.086396 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4982b9-3078-4d68-b63b-1e107f20aa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:46.087674 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4982b9-3078-4d68-b63b-1e107f20aa43 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:46.088081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4982b9-3078-4d68-b63b-1e107f20aa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:46.088593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4982b9-3078-4d68-b63b-1e107f20aa43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:46.089355 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4982b9-3078-4d68-b63b-1e107f20aa43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:46.090992 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4982b9-3078-4d68-b63b-1e107f20aa43 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:46.101824 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1396/2767] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:46 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:52:46.107288 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a862b6-8d53-43b2-962f-1a2936083365 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:46.124025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a862b6-8d53-43b2-962f-1a2936083365 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:46.130765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfce721-fe79-4b16-8d55-e8db46c75e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:46.131716 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfce721-fe79-4b16-8d55-e8db46c75e52 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:46.132137 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfce721-fe79-4b16-8d55-e8db46c75e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:46.132527 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfce721-fe79-4b16-8d55-e8db46c75e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:46.133593 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfce721-fe79-4b16-8d55-e8db46c75e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:46.136524 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfce721-fe79-4b16-8d55-e8db46c75e52 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:46.147910 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1372/2768] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:46 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:52:46.154166 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4982b9-3078-4d68-b63b-1e107f20aa43 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:46.169655 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4982b9-3078-4d68-b63b-1e107f20aa43 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:46.174960 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d762f-1bae-4cbe-9fe4-7f545ea1c6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:46.175709 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d762f-1bae-4cbe-9fe4-7f545ea1c6d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:46.176055 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d762f-1bae-4cbe-9fe4-7f545ea1c6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:46.176386 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d762f-1bae-4cbe-9fe4-7f545ea1c6d5 admin admin] PATH_INFO: `/v3/projects/557cbe65acb94a749a05118b3f3b70ff/users/0ea7a204b1d34f7dacfe9f468f565e65/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:46.193378 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d762f-1bae-4cbe-9fe4-7f545ea1c6d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=557cbe65acb94a749a05118b3f3b70ff, user_id=0ea7a204b1d34f7dacfe9f468f565e65, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:46.196108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d762f-1bae-4cbe-9fe4-7f545ea1c6d5 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:46.211650 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1397/2769] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:52:46 2022] PUT /identity/v3/projects/557cbe65acb94a749a05118b3f3b70ff/users/0ea7a204b1d34f7dacfe9f468f565e65/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:46.217061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b98ab5-22f1-4cc6-b2fa-f2ee87eed49f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:46.217385 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b98ab5-22f1-4cc6-b2fa-f2ee87eed49f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:46.217669 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b98ab5-22f1-4cc6-b2fa-f2ee87eed49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:46.239389 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-63b98ab5-22f1-4cc6-b2fa-f2ee87eed49f None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:46.240670 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-63b98ab5-22f1-4cc6-b2fa-f2ee87eed49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:46.291667 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:46.303796 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1373/2770] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:46 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:46.314364 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d762f-1bae-4cbe-9fe4-7f545ea1c6d5 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:46.316285 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c17d762f-1bae-4cbe-9fe4-7f545ea1c6d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:46.333301 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:46.393927 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d762f-1bae-4cbe-9fe4-7f545ea1c6d5 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:46.397655 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee2fb1-71ae-49c5-bbb5-44cec31b34ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:46.398279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee2fb1-71ae-49c5-bbb5-44cec31b34ab service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:46.398562 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee2fb1-71ae-49c5-bbb5-44cec31b34ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:46.398843 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee2fb1-71ae-49c5-bbb5-44cec31b34ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:46.408835 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee2fb1-71ae-49c5-bbb5-44cec31b34ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:46.410343 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee2fb1-71ae-49c5-bbb5-44cec31b34ab service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:46.421519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1398/2771] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:52:46 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:51.943113 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e24d8-557b-472f-b66b-86f293516ad0 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:51.943698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e24d8-557b-472f-b66b-86f293516ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:51.943698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e24d8-557b-472f-b66b-86f293516ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:51.982949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-fd0e24d8-557b-472f-b66b-86f293516ad0 None None] MFA Rules not processed for user `0ea7a204b1d34f7dacfe9f468f565e65`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:51.984344 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fd0e24d8-557b-472f-b66b-86f293516ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:52.084949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1374/2772] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:52.090964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee2fb1-71ae-49c5-bbb5-44cec31b34ab service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:52.101792 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee2fb1-71ae-49c5-bbb5-44cec31b34ab service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:52.107588 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c585449-0d9c-4039-93c4-786be622800a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:52.108406 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c585449-0d9c-4039-93c4-786be622800a admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:52.108762 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c585449-0d9c-4039-93c4-786be622800a admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:52.109133 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c585449-0d9c-4039-93c4-786be622800a admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:52.110140 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c585449-0d9c-4039-93c4-786be622800a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:52.112187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c585449-0d9c-4039-93c4-786be622800a admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:52.156506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1399/2773] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:52:52 2022] POST /identity/v3/projects => generated 386 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:52:52.160862 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e24d8-557b-472f-b66b-86f293516ad0 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:52.176957 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e24d8-557b-472f-b66b-86f293516ad0 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:52.251009 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44c72f-7968-4fdb-932d-6fbb4ebbeb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:52.251657 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44c72f-7968-4fdb-932d-6fbb4ebbeb48 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:52.251941 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44c72f-7968-4fdb-932d-6fbb4ebbeb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:52.252217 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44c72f-7968-4fdb-932d-6fbb4ebbeb48 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:52.252907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44c72f-7968-4fdb-932d-6fbb4ebbeb48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:52.255348 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44c72f-7968-4fdb-932d-6fbb4ebbeb48 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:52.286658 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1375/2774] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:52:52 2022] POST /identity/v3/users => generated 300 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:52:52.292885 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c585449-0d9c-4039-93c4-786be622800a admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:52.306347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c585449-0d9c-4039-93c4-786be622800a admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:52.311336 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6da91-0afe-4867-8e1d-9f6d5ba2a855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:52.311962 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6da91-0afe-4867-8e1d-9f6d5ba2a855 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:52.312245 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6da91-0afe-4867-8e1d-9f6d5ba2a855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:52.312703 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6da91-0afe-4867-8e1d-9f6d5ba2a855 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:52.314006 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6da91-0afe-4867-8e1d-9f6d5ba2a855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:52.315956 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6da91-0afe-4867-8e1d-9f6d5ba2a855 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:52.329677 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1400/2775] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:52 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:52:52.334254 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44c72f-7968-4fdb-932d-6fbb4ebbeb48 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:52.345339 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44c72f-7968-4fdb-932d-6fbb4ebbeb48 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:52.352320 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b6436-61c6-4b1f-9b87-845e5a7e0cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:52.353535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b6436-61c6-4b1f-9b87-845e5a7e0cc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:52.354022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b6436-61c6-4b1f-9b87-845e5a7e0cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:52.354499 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b6436-61c6-4b1f-9b87-845e5a7e0cc8 admin admin] PATH_INFO: `/v3/projects/da3b6337b1924f6c95d749b0e4ef17d1/users/273a79f0fa4e4b1eb3f36866396a34a9/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:52.379002 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b6436-61c6-4b1f-9b87-845e5a7e0cc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da3b6337b1924f6c95d749b0e4ef17d1, user_id=273a79f0fa4e4b1eb3f36866396a34a9, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:52.381869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b6436-61c6-4b1f-9b87-845e5a7e0cc8 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:52.398764 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1376/2776] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:52:52 2022] PUT /identity/v3/projects/da3b6337b1924f6c95d749b0e4ef17d1/users/273a79f0fa4e4b1eb3f36866396a34a9/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:52:52.403694 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6da91-0afe-4867-8e1d-9f6d5ba2a855 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:52.414715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6da91-0afe-4867-8e1d-9f6d5ba2a855 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:52.480576 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94d423-865e-4519-8f2c-03b96e520f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:52.481228 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94d423-865e-4519-8f2c-03b96e520f6d admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:52.481462 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94d423-865e-4519-8f2c-03b96e520f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:52.481688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94d423-865e-4519-8f2c-03b96e520f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:52.482392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94d423-865e-4519-8f2c-03b96e520f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:52.484714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94d423-865e-4519-8f2c-03b96e520f6d admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:52.499054 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1401/2777] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:52:52 2022] GET /identity/v3/roles => generated 1427 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:52:52.503858 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b6436-61c6-4b1f-9b87-845e5a7e0cc8 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:52.515816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b6436-61c6-4b1f-9b87-845e5a7e0cc8 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:52.520884 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf42e8-f653-4ac0-a442-4dc883f04937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:52.521704 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf42e8-f653-4ac0-a442-4dc883f04937 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:52.521990 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf42e8-f653-4ac0-a442-4dc883f04937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:52.522272 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf42e8-f653-4ac0-a442-4dc883f04937 admin admin] PATH_INFO: `/v3/projects/da3b6337b1924f6c95d749b0e4ef17d1/users/273a79f0fa4e4b1eb3f36866396a34a9/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:52.524984 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf42e8-f653-4ac0-a442-4dc883f04937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da3b6337b1924f6c95d749b0e4ef17d1, user_id=273a79f0fa4e4b1eb3f36866396a34a9, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:52.527834 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf42e8-f653-4ac0-a442-4dc883f04937 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:52.547006 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1377/2778] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:52:52 2022] PUT /identity/v3/projects/da3b6337b1924f6c95d749b0e4ef17d1/users/273a79f0fa4e4b1eb3f36866396a34a9/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:52:57.969948 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa0d46-0a53-4f14-a930-b84b17e3ca1c None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:57.970516 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa0d46-0a53-4f14-a930-b84b17e3ca1c None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:57.970763 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa0d46-0a53-4f14-a930-b84b17e3ca1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:58.013604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-a8aa0d46-0a53-4f14-a930-b84b17e3ca1c None None] MFA Rules not processed for user `273a79f0fa4e4b1eb3f36866396a34a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:58.015114 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-a8aa0d46-0a53-4f14-a930-b84b17e3ca1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:58.119229 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1402/2779] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:52:57 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:58.132939 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf42e8-f653-4ac0-a442-4dc883f04937 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:58.137865 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-beaf42e8-f653-4ac0-a442-4dc883f04937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:58.193930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:58.226336 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf42e8-f653-4ac0-a442-4dc883f04937 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:58.231265 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be7eb7-1947-40bc-9d97-3f8cd456cdfb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd959ddd087724547a11c3d4c796d8ab2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:58.232083 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be7eb7-1947-40bc-9d97-3f8cd456cdfb service glance] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:58.232415 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be7eb7-1947-40bc-9d97-3f8cd456cdfb service glance] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:58.232647 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be7eb7-1947-40bc-9d97-3f8cd456cdfb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:58.351891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be7eb7-1947-40bc-9d97-3f8cd456cdfb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:58.354571 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be7eb7-1947-40bc-9d97-3f8cd456cdfb service glance] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:58.366127 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1378/2780] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:52:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:52:58.542123 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df5d2c-e9d7-4a12-9221-c5b7f07bc4df None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:58.542682 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df5d2c-e9d7-4a12-9221-c5b7f07bc4df None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:58.543253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df5d2c-e9d7-4a12-9221-c5b7f07bc4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:58.560030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:58.599639 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-07df5d2c-e9d7-4a12-9221-c5b7f07bc4df None None] MFA Rules not processed for user `1cd36b543bd9487ebd8ce578e9a76c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:58.600903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-07df5d2c-e9d7-4a12-9221-c5b7f07bc4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:58.704595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1403/2781] 10.222.0.100 () {58 vars in 1014 bytes} [Tue Aug 2 09:52:58 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:58.772509 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be7eb7-1947-40bc-9d97-3f8cd456cdfb service glance] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:58.774569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-43be7eb7-1947-40bc-9d97-3f8cd456cdfb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:58.859936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be7eb7-1947-40bc-9d97-3f8cd456cdfb service glance] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:58.865500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e1240-654d-48db-956b-c32caabca2f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50cb532b58ed4951ba960a0a7d579b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:58.865920 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e1240-654d-48db-956b-c32caabca2f5 service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:58.866383 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e1240-654d-48db-956b-c32caabca2f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:58.866606 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e1240-654d-48db-956b-c32caabca2f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:58.876186 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e1240-654d-48db-956b-c32caabca2f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:58.877847 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e1240-654d-48db-956b-c32caabca2f5 service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:58.887508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1379/2782] 10.222.0.100 () {60 vars in 1389 bytes} [Tue Aug 2 09:52:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 02 09:53:08.100818 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac7165-c95f-4789-8520-69b84474548a None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:08.101465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac7165-c95f-4789-8520-69b84474548a None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:08.101465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac7165-c95f-4789-8520-69b84474548a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:08.130053 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-d3ac7165-c95f-4789-8520-69b84474548a None None] MFA Rules not processed for user `0ea7a204b1d34f7dacfe9f468f565e65`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:08.131710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d3ac7165-c95f-4789-8520-69b84474548a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:08.139338 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1404/2783] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:08.884262 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=63 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:10.772919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbd799-ffbf-4ab2-b7c8-e4e07cc02036 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:10.774170 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbd799-ffbf-4ab2-b7c8-e4e07cc02036 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:10.774345 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbd799-ffbf-4ab2-b7c8-e4e07cc02036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:10.796279 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b2cbd799-ffbf-4ab2-b7c8-e4e07cc02036 None None] MFA Rules not processed for user `1cd36b543bd9487ebd8ce578e9a76c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:10.797595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b2cbd799-ffbf-4ab2-b7c8-e4e07cc02036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:10.806925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1380/2784] 10.222.0.100 () {58 vars in 1014 bytes} [Tue Aug 2 09:53:10 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:53:10.825841 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac7165-c95f-4789-8520-69b84474548a None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:10.847267 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac7165-c95f-4789-8520-69b84474548a None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:10.855106 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ade8a3-eee1-4f5b-9eb4-fb6a66424bd3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50cb532b58ed4951ba960a0a7d579b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:10.856160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ade8a3-eee1-4f5b-9eb4-fb6a66424bd3 service swift] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:10.856637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ade8a3-eee1-4f5b-9eb4-fb6a66424bd3 service swift] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:10.857669 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ade8a3-eee1-4f5b-9eb4-fb6a66424bd3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:10.872736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ade8a3-eee1-4f5b-9eb4-fb6a66424bd3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:10.874347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ade8a3-eee1-4f5b-9eb4-fb6a66424bd3 service swift] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:10.884284 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1405/2785] 10.222.0.100 () {60 vars in 1389 bytes} [Tue Aug 2 09:53:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:53:20.789859 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:20.879990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=35 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:21.418144 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbd799-ffbf-4ab2-b7c8-e4e07cc02036 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:21.419685 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b2cbd799-ffbf-4ab2-b7c8-e4e07cc02036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:21.519790 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbd799-ffbf-4ab2-b7c8-e4e07cc02036 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:21.527287 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206edd4d-4f8c-431f-a28c-32a04e653ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:21.528360 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206edd4d-4f8c-431f-a28c-32a04e653ffc service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:21.529037 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206edd4d-4f8c-431f-a28c-32a04e653ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:21.529312 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206edd4d-4f8c-431f-a28c-32a04e653ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:21.536051 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-206edd4d-4f8c-431f-a28c-32a04e653ffc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:21.553799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206edd4d-4f8c-431f-a28c-32a04e653ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:21.556487 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206edd4d-4f8c-431f-a28c-32a04e653ffc service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:21.571997 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1381/2786] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:53:21 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:53:30.592069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ade8a3-eee1-4f5b-9eb4-fb6a66424bd3 service swift] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:30.602413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ade8a3-eee1-4f5b-9eb4-fb6a66424bd3 service swift] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:30.606125 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71fc3b-3ef7-4dc1-96d3-7907eca3ce41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:30.606734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71fc3b-3ef7-4dc1-96d3-7907eca3ce41 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:30.607020 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71fc3b-3ef7-4dc1-96d3-7907eca3ce41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:30.607301 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71fc3b-3ef7-4dc1-96d3-7907eca3ce41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:30.609034 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1f71fc3b-3ef7-4dc1-96d3-7907eca3ce41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:30.678657 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f71fc3b-3ef7-4dc1-96d3-7907eca3ce41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:30.680408 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f71fc3b-3ef7-4dc1-96d3-7907eca3ce41 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:30.690956 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1406/2787] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:53:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:31.564393 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=26 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:34.205413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206edd4d-4f8c-431f-a28c-32a04e653ffc service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:34.208927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-206edd4d-4f8c-431f-a28c-32a04e653ffc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:34.287654 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71fc3b-3ef7-4dc1-96d3-7907eca3ce41 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:34.289710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1f71fc3b-3ef7-4dc1-96d3-7907eca3ce41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:34.313927 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71fc3b-3ef7-4dc1-96d3-7907eca3ce41 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:34.325664 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baef854-f416-404d-afd1-0ea8c8721d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:34.326918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baef854-f416-404d-afd1-0ea8c8721d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:34.327310 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baef854-f416-404d-afd1-0ea8c8721d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:34.328898 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baef854-f416-404d-afd1-0ea8c8721d2c admin admin] PATH_INFO: `/v3/users/18728d1c3451497480556461b4ec80d5` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:34.335333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baef854-f416-404d-afd1-0ea8c8721d2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=18728d1c3451497480556461b4ec80d5)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:34.339028 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baef854-f416-404d-afd1-0ea8c8721d2c admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:34.339897 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206edd4d-4f8c-431f-a28c-32a04e653ffc service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:34.347697 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31836bc-b895-4184-90e7-67acc4ef7c4b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b40981fe179a43dc81553d3594dad649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:34.348345 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31836bc-b895-4184-90e7-67acc4ef7c4b service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:34.349508 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31836bc-b895-4184-90e7-67acc4ef7c4b service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:34.349876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31836bc-b895-4184-90e7-67acc4ef7c4b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:34.352356 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d31836bc-b895-4184-90e7-67acc4ef7c4b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:34.371260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31836bc-b895-4184-90e7-67acc4ef7c4b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:34.373454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31836bc-b895-4184-90e7-67acc4ef7c4b service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:34.398217 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1382/2788] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:53:34 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:34.399499 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31836bc-b895-4184-90e7-67acc4ef7c4b service placement] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:34.401275 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d31836bc-b895-4184-90e7-67acc4ef7c4b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:34.444854 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-7baef854-f416-404d-afd1-0ea8c8721d2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18728d1c3451497480556461b4ec80d5'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:34.446282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-7baef854-f416-404d-afd1-0ea8c8721d2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18728d1c3451497480556461b4ec80d5'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:34.459630 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-7baef854-f416-404d-afd1-0ea8c8721d2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18728d1c3451497480556461b4ec80d5'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:34.482580 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-7baef854-f416-404d-afd1-0ea8c8721d2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18728d1c3451497480556461b4ec80d5'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:34.490178 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-7baef854-f416-404d-afd1-0ea8c8721d2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18728d1c3451497480556461b4ec80d5'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:34.494327 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31836bc-b895-4184-90e7-67acc4ef7c4b service placement] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:34.498808 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77afb6-dbdf-44f5-a983-74823f44327a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:34.499481 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77afb6-dbdf-44f5-a983-74823f44327a service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:34.499767 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77afb6-dbdf-44f5-a983-74823f44327a service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:34.500050 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77afb6-dbdf-44f5-a983-74823f44327a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:34.500296 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1407/2789] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:53:34 2022] DELETE /identity/v3/users/18728d1c3451497480556461b4ec80d5 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:53:34.504382 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4d77afb6-dbdf-44f5-a983-74823f44327a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:34.602728 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d77afb6-dbdf-44f5-a983-74823f44327a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:34.604187 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d77afb6-dbdf-44f5-a983-74823f44327a service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:34.634282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1383/2790] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:53:34 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:34.727076 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baef854-f416-404d-afd1-0ea8c8721d2c admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:34.728849 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-7baef854-f416-404d-afd1-0ea8c8721d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:34.854059 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baef854-f416-404d-afd1-0ea8c8721d2c admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:34.859288 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e671af8-aeb2-4944-81f5-dc8f8eadbdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:34.860009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e671af8-aeb2-4944-81f5-dc8f8eadbdaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:34.860393 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e671af8-aeb2-4944-81f5-dc8f8eadbdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:34.860724 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e671af8-aeb2-4944-81f5-dc8f8eadbdaa admin admin] PATH_INFO: `/v3/projects/e2757dae0d3a45a29427faed00e98996` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:34.862315 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e671af8-aeb2-4944-81f5-dc8f8eadbdaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2757dae0d3a45a29427faed00e98996)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:34.864659 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e671af8-aeb2-4944-81f5-dc8f8eadbdaa admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:34.883560 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:34.962044 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-7e671af8-aeb2-4944-81f5-dc8f8eadbdaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2757dae0d3a45a29427faed00e98996', 'request_id': 'req-7e671af8-aeb2-4944-81f5-dc8f8eadbdaa'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:34.974152 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-7e671af8-aeb2-4944-81f5-dc8f8eadbdaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2757dae0d3a45a29427faed00e98996', 'request_id': 'req-7e671af8-aeb2-4944-81f5-dc8f8eadbdaa'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:34.997713 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-7e671af8-aeb2-4944-81f5-dc8f8eadbdaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2757dae0d3a45a29427faed00e98996', 'request_id': 'req-7e671af8-aeb2-4944-81f5-dc8f8eadbdaa'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:35.041831 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-7e671af8-aeb2-4944-81f5-dc8f8eadbdaa admin admin] The token cache is being invalidate because project e2757dae0d3a45a29427faed00e98996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:53:35.042450 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-7e671af8-aeb2-4944-81f5-dc8f8eadbdaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:35.045526 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-7e671af8-aeb2-4944-81f5-dc8f8eadbdaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:35.047861 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1408/2791] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:53:34 2022] DELETE /identity/v3/projects/e2757dae0d3a45a29427faed00e98996 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:35.055468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4eeff0-b270-43e2-a3d5-f5de279163e3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:35.055907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4eeff0-b270-43e2-a3d5-f5de279163e3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:35.056228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4eeff0-b270-43e2-a3d5-f5de279163e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:35.086222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ce4eeff0-b270-43e2-a3d5-f5de279163e3 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:35.089623 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ce4eeff0-b270-43e2-a3d5-f5de279163e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:35.113620 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:35.202074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1384/2792] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:35.205794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc6dbd-a75b-4931-9d2a-06617163563d None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:35.206117 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc6dbd-a75b-4931-9d2a-06617163563d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:35.206397 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc6dbd-a75b-4931-9d2a-06617163563d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:35.230603 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-0cfc6dbd-a75b-4931-9d2a-06617163563d None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:35.231811 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-0cfc6dbd-a75b-4931-9d2a-06617163563d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:35.242346 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1409/2793] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:53:35.247664 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4eeff0-b270-43e2-a3d5-f5de279163e3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:35.264154 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4eeff0-b270-43e2-a3d5-f5de279163e3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:35.269070 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acb7f1-9e44-4d2b-8e8e-65dfc4296beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:35.269735 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acb7f1-9e44-4d2b-8e8e-65dfc4296beb admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:35.270020 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acb7f1-9e44-4d2b-8e8e-65dfc4296beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:35.270299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acb7f1-9e44-4d2b-8e8e-65dfc4296beb admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:35.271078 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acb7f1-9e44-4d2b-8e8e-65dfc4296beb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:53:35.271315 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acb7f1-9e44-4d2b-8e8e-65dfc4296beb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:35.273814 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acb7f1-9e44-4d2b-8e8e-65dfc4296beb admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:35.290391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1385/2794] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:53:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:53:35.294508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc6dbd-a75b-4931-9d2a-06617163563d None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:35.306257 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc6dbd-a75b-4931-9d2a-06617163563d None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:35.317928 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577b949-6e6b-4157-9922-3153ae6eca90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:35.317928 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577b949-6e6b-4157-9922-3153ae6eca90 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:35.318279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577b949-6e6b-4157-9922-3153ae6eca90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:35.318279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577b949-6e6b-4157-9922-3153ae6eca90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:35.318952 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577b949-6e6b-4157-9922-3153ae6eca90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:35.321198 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577b949-6e6b-4157-9922-3153ae6eca90 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:35.365695 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1410/2795] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:53:35 2022] POST /identity/v3/projects => generated 374 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:53:35.370727 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acb7f1-9e44-4d2b-8e8e-65dfc4296beb admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:35.387975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acb7f1-9e44-4d2b-8e8e-65dfc4296beb admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:35.467513 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf57dc-3d24-40a6-8ee7-5c87b6b77180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:35.468046 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf57dc-3d24-40a6-8ee7-5c87b6b77180 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:35.468329 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf57dc-3d24-40a6-8ee7-5c87b6b77180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:35.468600 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf57dc-3d24-40a6-8ee7-5c87b6b77180 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:35.469315 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf57dc-3d24-40a6-8ee7-5c87b6b77180 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:35.471316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf57dc-3d24-40a6-8ee7-5c87b6b77180 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:35.498076 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1386/2796] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:53:35 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:53:35.503593 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577b949-6e6b-4157-9922-3153ae6eca90 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:35.516104 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577b949-6e6b-4157-9922-3153ae6eca90 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:35.523986 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2650d0-39b2-4e8b-b9a7-bbd76272c088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:35.524692 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2650d0-39b2-4e8b-b9a7-bbd76272c088 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:35.525047 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2650d0-39b2-4e8b-b9a7-bbd76272c088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:35.525421 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2650d0-39b2-4e8b-b9a7-bbd76272c088 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:35.526297 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2650d0-39b2-4e8b-b9a7-bbd76272c088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:35.528058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2650d0-39b2-4e8b-b9a7-bbd76272c088 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:35.543059 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1411/2797] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:53:35 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:53:35.548097 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf57dc-3d24-40a6-8ee7-5c87b6b77180 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:35.561542 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf57dc-3d24-40a6-8ee7-5c87b6b77180 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:35.567575 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097ac0c-a6da-403c-8489-a51b079fce81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:35.568272 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097ac0c-a6da-403c-8489-a51b079fce81 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:35.568557 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097ac0c-a6da-403c-8489-a51b079fce81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:35.569580 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097ac0c-a6da-403c-8489-a51b079fce81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:35.570523 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097ac0c-a6da-403c-8489-a51b079fce81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:35.572454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097ac0c-a6da-403c-8489-a51b079fce81 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:35.589002 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1387/2798] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:53:35 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:53:35.594588 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2650d0-39b2-4e8b-b9a7-bbd76272c088 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:35.610755 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2650d0-39b2-4e8b-b9a7-bbd76272c088 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:35.615699 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1363b1-efc7-47f2-bdfc-66ee2d450154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:35.616469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1363b1-efc7-47f2-bdfc-66ee2d450154 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:35.616773 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1363b1-efc7-47f2-bdfc-66ee2d450154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:35.617333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1363b1-efc7-47f2-bdfc-66ee2d450154 admin admin] PATH_INFO: `/v3/projects/4086822d1c7442d8bf40fb3528c530d1/users/8eacaef97e7c4ce3836c2a1185a5f5f7/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:35.636404 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1363b1-efc7-47f2-bdfc-66ee2d450154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4086822d1c7442d8bf40fb3528c530d1, user_id=8eacaef97e7c4ce3836c2a1185a5f5f7, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:35.639320 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1363b1-efc7-47f2-bdfc-66ee2d450154 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:35.656680 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1412/2799] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:53:35 2022] PUT /identity/v3/projects/4086822d1c7442d8bf40fb3528c530d1/users/8eacaef97e7c4ce3836c2a1185a5f5f7/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:53:35.671119 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0f694-0b5d-47bd-8958-a4780c26ec4a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:35.671569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0f694-0b5d-47bd-8958-a4780c26ec4a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:35.671831 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0f694-0b5d-47bd-8958-a4780c26ec4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:35.711819 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-76a0f694-0b5d-47bd-8958-a4780c26ec4a None None] MFA Rules not processed for user `8eacaef97e7c4ce3836c2a1185a5f5f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:35.713549 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-76a0f694-0b5d-47bd-8958-a4780c26ec4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:35.797226 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1388/2800] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:53:35.802411 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1363b1-efc7-47f2-bdfc-66ee2d450154 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:35.889842 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1363b1-efc7-47f2-bdfc-66ee2d450154 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:35.895108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fca4d6-f4a2-4316-b6f3-305094ffdf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:35.895667 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fca4d6-f4a2-4316-b6f3-305094ffdf0e admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:35.895953 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fca4d6-f4a2-4316-b6f3-305094ffdf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:35.896232 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fca4d6-f4a2-4316-b6f3-305094ffdf0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:35.896909 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fca4d6-f4a2-4316-b6f3-305094ffdf0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:35.899410 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fca4d6-f4a2-4316-b6f3-305094ffdf0e admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:35.936478 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1413/2801] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:53:35 2022] POST /identity/v3/projects => generated 374 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:53:35.937880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0f694-0b5d-47bd-8958-a4780c26ec4a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:35.951115 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0f694-0b5d-47bd-8958-a4780c26ec4a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:35.966649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:36.023321 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa08e8b-c913-4fd4-ac9b-236cac085ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:36.023952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa08e8b-c913-4fd4-ac9b-236cac085ac0 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:36.024241 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa08e8b-c913-4fd4-ac9b-236cac085ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:36.024521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa08e8b-c913-4fd4-ac9b-236cac085ac0 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:36.025233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa08e8b-c913-4fd4-ac9b-236cac085ac0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:36.027150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa08e8b-c913-4fd4-ac9b-236cac085ac0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:36.050619 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1389/2802] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:53:35 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:53:36.055162 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fca4d6-f4a2-4316-b6f3-305094ffdf0e admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:36.068510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fca4d6-f4a2-4316-b6f3-305094ffdf0e admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:36.074380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1349c-4858-4f8c-a760-25c933a213df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:36.075218 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1349c-4858-4f8c-a760-25c933a213df admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:36.075614 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1349c-4858-4f8c-a760-25c933a213df admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:36.075990 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1349c-4858-4f8c-a760-25c933a213df admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:36.076851 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea1349c-4858-4f8c-a760-25c933a213df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:36.078975 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea1349c-4858-4f8c-a760-25c933a213df admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:36.092449 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1414/2803] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:53:36 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:53:36.097809 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa08e8b-c913-4fd4-ac9b-236cac085ac0 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:36.111278 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa08e8b-c913-4fd4-ac9b-236cac085ac0 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:36.117466 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc8584-2049-44c9-88f6-82f671e01536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:36.118150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc8584-2049-44c9-88f6-82f671e01536 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:36.118429 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc8584-2049-44c9-88f6-82f671e01536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:36.118705 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc8584-2049-44c9-88f6-82f671e01536 admin admin] PATH_INFO: `/v3/projects/01535b56edb245af87cb88cdc7d70228/users/2aaa5ec93088455baf7aaaf48fc6a4d9/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:36.135646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc8584-2049-44c9-88f6-82f671e01536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01535b56edb245af87cb88cdc7d70228, user_id=2aaa5ec93088455baf7aaaf48fc6a4d9, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:36.138398 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc8584-2049-44c9-88f6-82f671e01536 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:36.154552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1390/2804] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:53:36 2022] PUT /identity/v3/projects/01535b56edb245af87cb88cdc7d70228/users/2aaa5ec93088455baf7aaaf48fc6a4d9/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:36.159027 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1349c-4858-4f8c-a760-25c933a213df admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:36.170179 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1349c-4858-4f8c-a760-25c933a213df admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:36.173901 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:36.233923 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90af13-01de-4040-aa14-373be7d968a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:36.234542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90af13-01de-4040-aa14-373be7d968a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:36.234823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90af13-01de-4040-aa14-373be7d968a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:36.235097 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90af13-01de-4040-aa14-373be7d968a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:36.236105 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90af13-01de-4040-aa14-373be7d968a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:36.238366 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90af13-01de-4040-aa14-373be7d968a9 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:36.248490 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1415/2805] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:53:36 2022] GET /identity/v3/roles => generated 1427 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:53:36.254096 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc8584-2049-44c9-88f6-82f671e01536 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:36.265390 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc8584-2049-44c9-88f6-82f671e01536 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:36.270420 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67524ccb-8180-4e34-879e-41a061d5ba79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:36.271079 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67524ccb-8180-4e34-879e-41a061d5ba79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:36.271368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67524ccb-8180-4e34-879e-41a061d5ba79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:36.271648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67524ccb-8180-4e34-879e-41a061d5ba79 admin admin] PATH_INFO: `/v3/projects/01535b56edb245af87cb88cdc7d70228/users/2aaa5ec93088455baf7aaaf48fc6a4d9/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:36.274303 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67524ccb-8180-4e34-879e-41a061d5ba79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01535b56edb245af87cb88cdc7d70228, user_id=2aaa5ec93088455baf7aaaf48fc6a4d9, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:36.276938 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67524ccb-8180-4e34-879e-41a061d5ba79 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:36.288025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1391/2806] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:53:36 2022] PUT /identity/v3/projects/01535b56edb245af87cb88cdc7d70228/users/2aaa5ec93088455baf7aaaf48fc6a4d9/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:53:36.298283 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740952c-0893-441a-94d0-2ce22387df25 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:36.299794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740952c-0893-441a-94d0-2ce22387df25 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:36.299794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740952c-0893-441a-94d0-2ce22387df25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:36.327879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-e740952c-0893-441a-94d0-2ce22387df25 None None] MFA Rules not processed for user `2aaa5ec93088455baf7aaaf48fc6a4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:36.329010 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e740952c-0893-441a-94d0-2ce22387df25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:36.404214 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1416/2807] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:36.418056 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67524ccb-8180-4e34-879e-41a061d5ba79 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:36.419866 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-67524ccb-8180-4e34-879e-41a061d5ba79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:36.489936 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67524ccb-8180-4e34-879e-41a061d5ba79 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:36.496014 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eab693-3510-465a-9b57-058ff791932c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:36.496937 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eab693-3510-465a-9b57-058ff791932c service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:36.497380 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eab693-3510-465a-9b57-058ff791932c service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:36.497665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eab693-3510-465a-9b57-058ff791932c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:36.511002 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eab693-3510-465a-9b57-058ff791932c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:36.512442 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eab693-3510-465a-9b57-058ff791932c service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:36.523750 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1392/2808] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:53:36 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:37.030316 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740952c-0893-441a-94d0-2ce22387df25 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:37.103585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740952c-0893-441a-94d0-2ce22387df25 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:37.109004 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054698f-4c1d-475b-a119-b8da0cf76c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:37.109688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054698f-4c1d-475b-a119-b8da0cf76c06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:37.109968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054698f-4c1d-475b-a119-b8da0cf76c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:37.110253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054698f-4c1d-475b-a119-b8da0cf76c06 admin admin] PATH_INFO: `/v3/users/8eacaef97e7c4ce3836c2a1185a5f5f7` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:37.111460 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054698f-4c1d-475b-a119-b8da0cf76c06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eacaef97e7c4ce3836c2a1185a5f5f7)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:37.114585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054698f-4c1d-475b-a119-b8da0cf76c06 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:37.159279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:37.182766 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-e054698f-4c1d-475b-a119-b8da0cf76c06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eacaef97e7c4ce3836c2a1185a5f5f7'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:37.183700 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-e054698f-4c1d-475b-a119-b8da0cf76c06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eacaef97e7c4ce3836c2a1185a5f5f7'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:37.199799 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-e054698f-4c1d-475b-a119-b8da0cf76c06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eacaef97e7c4ce3836c2a1185a5f5f7'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:37.221478 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-e054698f-4c1d-475b-a119-b8da0cf76c06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eacaef97e7c4ce3836c2a1185a5f5f7'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:37.233846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-e054698f-4c1d-475b-a119-b8da0cf76c06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eacaef97e7c4ce3836c2a1185a5f5f7'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:37.241305 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1417/2809] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:53:37 2022] DELETE /identity/v3/users/8eacaef97e7c4ce3836c2a1185a5f5f7 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:37.246330 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b21fd8-505d-4383-95d2-3e3689b95446 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:37.246660 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b21fd8-505d-4383-95d2-3e3689b95446 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:37.246973 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b21fd8-505d-4383-95d2-3e3689b95446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:37.266945 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-27b21fd8-505d-4383-95d2-3e3689b95446 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:37.268144 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-27b21fd8-505d-4383-95d2-3e3689b95446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:37.353443 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1393/2810] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:37 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:37.364165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054698f-4c1d-475b-a119-b8da0cf76c06 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:37.366740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e054698f-4c1d-475b-a119-b8da0cf76c06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:37.433898 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054698f-4c1d-475b-a119-b8da0cf76c06 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:37.437871 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe259e9-d1b4-4042-be83-b0d4cfa10035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:37.438502 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe259e9-d1b4-4042-be83-b0d4cfa10035 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:37.438786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe259e9-d1b4-4042-be83-b0d4cfa10035 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:37.439071 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe259e9-d1b4-4042-be83-b0d4cfa10035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:37.449977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe259e9-d1b4-4042-be83-b0d4cfa10035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:37.451439 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe259e9-d1b4-4042-be83-b0d4cfa10035 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:37.464597 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1418/2811] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:53:37 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:37.823875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b21fd8-505d-4383-95d2-3e3689b95446 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:37.825794 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-27b21fd8-505d-4383-95d2-3e3689b95446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:37.843904 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b21fd8-505d-4383-95d2-3e3689b95446 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:37.849206 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0b6d8-93d2-4299-8b44-ec192ef9b54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:37.849912 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0b6d8-93d2-4299-8b44-ec192ef9b54b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:37.850848 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0b6d8-93d2-4299-8b44-ec192ef9b54b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:37.851260 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0b6d8-93d2-4299-8b44-ec192ef9b54b admin admin] PATH_INFO: `/v3/projects/4086822d1c7442d8bf40fb3528c530d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:37.852849 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0b6d8-93d2-4299-8b44-ec192ef9b54b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4086822d1c7442d8bf40fb3528c530d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:37.855018 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0b6d8-93d2-4299-8b44-ec192ef9b54b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:37.865493 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:37.916649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-02c0b6d8-93d2-4299-8b44-ec192ef9b54b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4086822d1c7442d8bf40fb3528c530d1', 'request_id': 'req-02c0b6d8-93d2-4299-8b44-ec192ef9b54b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:37.923920 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-02c0b6d8-93d2-4299-8b44-ec192ef9b54b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4086822d1c7442d8bf40fb3528c530d1', 'request_id': 'req-02c0b6d8-93d2-4299-8b44-ec192ef9b54b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:37.933366 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-02c0b6d8-93d2-4299-8b44-ec192ef9b54b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4086822d1c7442d8bf40fb3528c530d1', 'request_id': 'req-02c0b6d8-93d2-4299-8b44-ec192ef9b54b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:37.945772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-02c0b6d8-93d2-4299-8b44-ec192ef9b54b admin admin] The token cache is being invalidate because project 4086822d1c7442d8bf40fb3528c530d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:53:37.946168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-02c0b6d8-93d2-4299-8b44-ec192ef9b54b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:37.947198 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-02c0b6d8-93d2-4299-8b44-ec192ef9b54b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:37.949883 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1394/2812] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:53:37 2022] DELETE /identity/v3/projects/4086822d1c7442d8bf40fb3528c530d1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:37.955740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe259e9-d1b4-4042-be83-b0d4cfa10035 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:37.972442 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe259e9-d1b4-4042-be83-b0d4cfa10035 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:38.043673 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458b24a-40fd-4386-9718-cc090f3487b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:38.044679 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458b24a-40fd-4386-9718-cc090f3487b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:38.045081 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458b24a-40fd-4386-9718-cc090f3487b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:38.045402 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458b24a-40fd-4386-9718-cc090f3487b0 admin admin] PATH_INFO: `/v3/users/2aaa5ec93088455baf7aaaf48fc6a4d9` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:38.046614 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458b24a-40fd-4386-9718-cc090f3487b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aaa5ec93088455baf7aaaf48fc6a4d9)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:38.048560 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458b24a-40fd-4386-9718-cc090f3487b0 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:38.111838 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b458b24a-40fd-4386-9718-cc090f3487b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aaa5ec93088455baf7aaaf48fc6a4d9'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:38.112503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b458b24a-40fd-4386-9718-cc090f3487b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aaa5ec93088455baf7aaaf48fc6a4d9'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:38.118059 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:38.123471 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b458b24a-40fd-4386-9718-cc090f3487b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aaa5ec93088455baf7aaaf48fc6a4d9'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:38.144358 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b458b24a-40fd-4386-9718-cc090f3487b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aaa5ec93088455baf7aaaf48fc6a4d9'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:38.154326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-b458b24a-40fd-4386-9718-cc090f3487b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aaa5ec93088455baf7aaaf48fc6a4d9'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:38.160730 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1419/2813] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:53:37 2022] DELETE /identity/v3/users/2aaa5ec93088455baf7aaaf48fc6a4d9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:38.469964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0b6d8-93d2-4299-8b44-ec192ef9b54b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:38.471620 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-02c0b6d8-93d2-4299-8b44-ec192ef9b54b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:38.489659 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0b6d8-93d2-4299-8b44-ec192ef9b54b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:38.550349 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0dc7ec-2498-4e06-8180-9e9ffded0c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:38.550998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dc7ec-2498-4e06-8180-9e9ffded0c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:38.551291 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dc7ec-2498-4e06-8180-9e9ffded0c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:38.551572 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dc7ec-2498-4e06-8180-9e9ffded0c66 admin admin] PATH_INFO: `/v3/projects/01535b56edb245af87cb88cdc7d70228` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:38.552737 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0dc7ec-2498-4e06-8180-9e9ffded0c66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01535b56edb245af87cb88cdc7d70228)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:38.555366 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0dc7ec-2498-4e06-8180-9e9ffded0c66 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:38.607625 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5d0dc7ec-2498-4e06-8180-9e9ffded0c66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01535b56edb245af87cb88cdc7d70228', 'request_id': 'req-5d0dc7ec-2498-4e06-8180-9e9ffded0c66'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:38.613790 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5d0dc7ec-2498-4e06-8180-9e9ffded0c66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01535b56edb245af87cb88cdc7d70228', 'request_id': 'req-5d0dc7ec-2498-4e06-8180-9e9ffded0c66'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:38.623714 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5d0dc7ec-2498-4e06-8180-9e9ffded0c66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01535b56edb245af87cb88cdc7d70228', 'request_id': 'req-5d0dc7ec-2498-4e06-8180-9e9ffded0c66'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:38.635038 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5d0dc7ec-2498-4e06-8180-9e9ffded0c66 admin admin] The token cache is being invalidate because project 01535b56edb245af87cb88cdc7d70228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:53:38.635333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5d0dc7ec-2498-4e06-8180-9e9ffded0c66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:38.636074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5d0dc7ec-2498-4e06-8180-9e9ffded0c66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:38.637636 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1395/2814] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:53:38 2022] DELETE /identity/v3/projects/01535b56edb245af87cb88cdc7d70228 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:53:38.644021 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26377964-1209-4d21-9b22-7f21c68896f4 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:38.644537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26377964-1209-4d21-9b22-7f21c68896f4 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:38.644537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26377964-1209-4d21-9b22-7f21c68896f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:38.666099 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-26377964-1209-4d21-9b22-7f21c68896f4 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:38.667185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-26377964-1209-4d21-9b22-7f21c68896f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:38.730607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1420/2815] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:38.738608 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b7a33-ee40-4020-9797-c4fb48f515ef None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:38.738910 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b7a33-ee40-4020-9797-c4fb48f515ef None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:38.739192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b7a33-ee40-4020-9797-c4fb48f515ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:38.759369 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f46b7a33-ee40-4020-9797-c4fb48f515ef None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:38.760704 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f46b7a33-ee40-4020-9797-c4fb48f515ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:38.769232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1396/2816] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:38.774629 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26377964-1209-4d21-9b22-7f21c68896f4 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:38.784537 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26377964-1209-4d21-9b22-7f21c68896f4 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:38.790057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273c237-01a4-43d2-adab-81aa27fe6ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:38.790674 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273c237-01a4-43d2-adab-81aa27fe6ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:38.790959 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273c237-01a4-43d2-adab-81aa27fe6ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:38.791240 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273c237-01a4-43d2-adab-81aa27fe6ef3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:38.791941 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273c237-01a4-43d2-adab-81aa27fe6ef3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:53:38.792251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273c237-01a4-43d2-adab-81aa27fe6ef3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:38.793906 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273c237-01a4-43d2-adab-81aa27fe6ef3 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:38.806018 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1421/2817] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:53:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:53:38.810817 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b7a33-ee40-4020-9797-c4fb48f515ef None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:38.820791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b7a33-ee40-4020-9797-c4fb48f515ef None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:38.825910 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48079f1-5052-4a50-9cfa-2e53c20887e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:38.826519 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48079f1-5052-4a50-9cfa-2e53c20887e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:38.826801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48079f1-5052-4a50-9cfa-2e53c20887e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:38.827076 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48079f1-5052-4a50-9cfa-2e53c20887e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:38.827746 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48079f1-5052-4a50-9cfa-2e53c20887e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:38.829723 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48079f1-5052-4a50-9cfa-2e53c20887e9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:38.859890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:38.869168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1397/2818] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:53:38 2022] POST /identity/v3/projects => generated 386 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:53:38.874353 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273c237-01a4-43d2-adab-81aa27fe6ef3 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:38.888803 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273c237-01a4-43d2-adab-81aa27fe6ef3 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:38.956192 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626e05b-3852-429b-a970-fdcc0b0a36f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:38.956823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626e05b-3852-429b-a970-fdcc0b0a36f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:38.957162 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626e05b-3852-429b-a970-fdcc0b0a36f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:38.957443 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626e05b-3852-429b-a970-fdcc0b0a36f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:38.958124 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0626e05b-3852-429b-a970-fdcc0b0a36f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:38.960122 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0626e05b-3852-429b-a970-fdcc0b0a36f9 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:38.982954 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1422/2819] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:53:38 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:53:38.987609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48079f1-5052-4a50-9cfa-2e53c20887e9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:38.998733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48079f1-5052-4a50-9cfa-2e53c20887e9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:39.004316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0657c0-e843-4508-aeff-30701dd807f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:39.004947 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0657c0-e843-4508-aeff-30701dd807f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:39.005301 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0657c0-e843-4508-aeff-30701dd807f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:39.005556 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0657c0-e843-4508-aeff-30701dd807f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:39.006251 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0657c0-e843-4508-aeff-30701dd807f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:39.008701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0657c0-e843-4508-aeff-30701dd807f4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:39.018979 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1398/2820] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:53:38 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:53:39.023395 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626e05b-3852-429b-a970-fdcc0b0a36f9 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:39.035936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626e05b-3852-429b-a970-fdcc0b0a36f9 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:39.040909 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eff8a1-f93e-4525-8190-5419cd057bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:39.041635 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eff8a1-f93e-4525-8190-5419cd057bbf admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:39.041921 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eff8a1-f93e-4525-8190-5419cd057bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:39.042193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eff8a1-f93e-4525-8190-5419cd057bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:39.042888 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eff8a1-f93e-4525-8190-5419cd057bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:39.044464 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eff8a1-f93e-4525-8190-5419cd057bbf admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:39.055866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1423/2821] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:53:39 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:53:39.060055 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0657c0-e843-4508-aeff-30701dd807f4 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:39.071853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0657c0-e843-4508-aeff-30701dd807f4 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:39.076814 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396f52e-0d5c-4855-a4ea-2ea8a31569b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:39.077519 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396f52e-0d5c-4855-a4ea-2ea8a31569b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:39.077803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396f52e-0d5c-4855-a4ea-2ea8a31569b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:39.078086 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396f52e-0d5c-4855-a4ea-2ea8a31569b0 admin admin] PATH_INFO: `/v3/projects/f33b3e4bc13647bc9bf0245653f8c277/users/c460374f86c84bfcab833b8ef79e1c0c/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:39.093205 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396f52e-0d5c-4855-a4ea-2ea8a31569b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f33b3e4bc13647bc9bf0245653f8c277, user_id=c460374f86c84bfcab833b8ef79e1c0c, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:39.095889 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396f52e-0d5c-4855-a4ea-2ea8a31569b0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:39.112526 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1399/2822] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:53:39 2022] PUT /identity/v3/projects/f33b3e4bc13647bc9bf0245653f8c277/users/c460374f86c84bfcab833b8ef79e1c0c/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:39.185218 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fa02e-ab69-4456-a20e-4f9b631cbaa7 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:39.185547 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fa02e-ab69-4456-a20e-4f9b631cbaa7 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:39.185973 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fa02e-ab69-4456-a20e-4f9b631cbaa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:39.215786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-195fa02e-ab69-4456-a20e-4f9b631cbaa7 None None] MFA Rules not processed for user `c460374f86c84bfcab833b8ef79e1c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:39.217012 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-195fa02e-ab69-4456-a20e-4f9b631cbaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:39.243019 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:39.292356 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1424/2823] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:39 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:39.298456 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396f52e-0d5c-4855-a4ea-2ea8a31569b0 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:39.393598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396f52e-0d5c-4855-a4ea-2ea8a31569b0 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:39.401846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a74bee-f96c-4b32-bbb3-c0f0b8bc5f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:39.402876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a74bee-f96c-4b32-bbb3-c0f0b8bc5f1e admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:39.403363 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a74bee-f96c-4b32-bbb3-c0f0b8bc5f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:39.403834 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a74bee-f96c-4b32-bbb3-c0f0b8bc5f1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:39.405003 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a74bee-f96c-4b32-bbb3-c0f0b8bc5f1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:39.408545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a74bee-f96c-4b32-bbb3-c0f0b8bc5f1e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:39.438666 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1400/2824] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:53:39 2022] POST /identity/v3/projects => generated 388 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:53:39.443164 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fa02e-ab69-4456-a20e-4f9b631cbaa7 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:39.455097 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fa02e-ab69-4456-a20e-4f9b631cbaa7 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:39.519908 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033204d5-bea2-47e7-865b-4f319c773d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:39.520536 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033204d5-bea2-47e7-865b-4f319c773d88 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:39.520814 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033204d5-bea2-47e7-865b-4f319c773d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:39.521088 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033204d5-bea2-47e7-865b-4f319c773d88 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:39.521794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033204d5-bea2-47e7-865b-4f319c773d88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:39.525764 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033204d5-bea2-47e7-865b-4f319c773d88 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:39.547607 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1425/2825] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:53:39 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:53:39.551576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a74bee-f96c-4b32-bbb3-c0f0b8bc5f1e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:39.564043 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a74bee-f96c-4b32-bbb3-c0f0b8bc5f1e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:39.570232 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d5bac-24ed-4b57-b1f1-42ad64960c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:39.570842 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d5bac-24ed-4b57-b1f1-42ad64960c9f admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:39.571126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d5bac-24ed-4b57-b1f1-42ad64960c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:39.572855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d5bac-24ed-4b57-b1f1-42ad64960c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:39.572855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d5bac-24ed-4b57-b1f1-42ad64960c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:39.573707 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d5bac-24ed-4b57-b1f1-42ad64960c9f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:39.586145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1401/2826] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:53:39 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:53:39.590264 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033204d5-bea2-47e7-865b-4f319c773d88 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:39.601621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033204d5-bea2-47e7-865b-4f319c773d88 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:39.606258 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81838c6a-11cd-40ca-8ba6-60f7a444bd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:39.606866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81838c6a-11cd-40ca-8ba6-60f7a444bd9f admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:39.607150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81838c6a-11cd-40ca-8ba6-60f7a444bd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:39.607426 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81838c6a-11cd-40ca-8ba6-60f7a444bd9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:39.608118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81838c6a-11cd-40ca-8ba6-60f7a444bd9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:39.609756 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81838c6a-11cd-40ca-8ba6-60f7a444bd9f admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:39.620503 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1426/2827] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:53:39 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:53:39.624949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d5bac-24ed-4b57-b1f1-42ad64960c9f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:39.636966 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d5bac-24ed-4b57-b1f1-42ad64960c9f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:39.643671 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6ea86-dae7-4cdd-bb38-c8b1ee6ad0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:39.644396 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6ea86-dae7-4cdd-bb38-c8b1ee6ad0cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:39.644678 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6ea86-dae7-4cdd-bb38-c8b1ee6ad0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:39.644960 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6ea86-dae7-4cdd-bb38-c8b1ee6ad0cd admin admin] PATH_INFO: `/v3/projects/e482a6a346724335b4cc8ee14f74b859/users/a72b13c6e66d417aa690a5e7b0d62918/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:39.665302 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6ea86-dae7-4cdd-bb38-c8b1ee6ad0cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e482a6a346724335b4cc8ee14f74b859, user_id=a72b13c6e66d417aa690a5e7b0d62918, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:39.668000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6ea86-dae7-4cdd-bb38-c8b1ee6ad0cd admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:39.683341 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1402/2828] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:53:39 2022] PUT /identity/v3/projects/e482a6a346724335b4cc8ee14f74b859/users/a72b13c6e66d417aa690a5e7b0d62918/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:39.703399 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5e0f8-f1d6-43b1-af4e-6b1b22c84cfe None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:39.703964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5e0f8-f1d6-43b1-af4e-6b1b22c84cfe None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:39.704107 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5e0f8-f1d6-43b1-af4e-6b1b22c84cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:39.734475 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-dcf5e0f8-f1d6-43b1-af4e-6b1b22c84cfe None None] MFA Rules not processed for user `a72b13c6e66d417aa690a5e7b0d62918`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:39.735700 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-dcf5e0f8-f1d6-43b1-af4e-6b1b22c84cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:39.808178 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1427/2829] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:39 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:39.814351 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6ea86-dae7-4cdd-bb38-c8b1ee6ad0cd admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:39.889984 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6ea86-dae7-4cdd-bb38-c8b1ee6ad0cd admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:39.894775 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99e210-cf30-477b-9b8d-99409b86cd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:39.895399 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99e210-cf30-477b-9b8d-99409b86cd88 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:39.895677 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99e210-cf30-477b-9b8d-99409b86cd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:39.895947 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99e210-cf30-477b-9b8d-99409b86cd88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:39.896603 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99e210-cf30-477b-9b8d-99409b86cd88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:39.900605 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99e210-cf30-477b-9b8d-99409b86cd88 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:39.926758 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:39.935876 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1403/2830] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:53:39 2022] POST /identity/v3/projects => generated 386 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:53:39.939426 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5e0f8-f1d6-43b1-af4e-6b1b22c84cfe None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:39.950764 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5e0f8-f1d6-43b1-af4e-6b1b22c84cfe None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:39.982649 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:40.014438 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457d0b8-1143-4ec9-ba03-766089761436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:40.015085 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457d0b8-1143-4ec9-ba03-766089761436 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:40.015368 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457d0b8-1143-4ec9-ba03-766089761436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:40.015645 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457d0b8-1143-4ec9-ba03-766089761436 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:40.016312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457d0b8-1143-4ec9-ba03-766089761436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:40.018314 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457d0b8-1143-4ec9-ba03-766089761436 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:40.042182 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1428/2831] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:53:39 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:53:40.046421 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99e210-cf30-477b-9b8d-99409b86cd88 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:40.057396 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99e210-cf30-477b-9b8d-99409b86cd88 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:40.062164 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4257001-8c56-4dcc-9c5b-c27afb273b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:40.062782 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4257001-8c56-4dcc-9c5b-c27afb273b97 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:40.063066 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4257001-8c56-4dcc-9c5b-c27afb273b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:40.063344 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4257001-8c56-4dcc-9c5b-c27afb273b97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:40.064034 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4257001-8c56-4dcc-9c5b-c27afb273b97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:40.065649 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4257001-8c56-4dcc-9c5b-c27afb273b97 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:40.077448 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1404/2832] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:53:40 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:53:40.082094 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457d0b8-1143-4ec9-ba03-766089761436 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:40.092955 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457d0b8-1143-4ec9-ba03-766089761436 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:40.100446 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4f2ac-4d69-4314-a62b-b3f555f2b606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:40.102239 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4f2ac-4d69-4314-a62b-b3f555f2b606 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:40.102897 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4f2ac-4d69-4314-a62b-b3f555f2b606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:40.103178 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4f2ac-4d69-4314-a62b-b3f555f2b606 admin admin] PATH_INFO: `/v3/projects/695d099ebda84ddeb89b40a81ba00114/users/400345afd2fc4969937164c442b3d550/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:40.118793 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4f2ac-4d69-4314-a62b-b3f555f2b606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=695d099ebda84ddeb89b40a81ba00114, user_id=400345afd2fc4969937164c442b3d550, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:40.121481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4f2ac-4d69-4314-a62b-b3f555f2b606 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:40.135855 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1429/2833] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:53:40 2022] PUT /identity/v3/projects/695d099ebda84ddeb89b40a81ba00114/users/400345afd2fc4969937164c442b3d550/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:40.140001 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4257001-8c56-4dcc-9c5b-c27afb273b97 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:40.157316 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4257001-8c56-4dcc-9c5b-c27afb273b97 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:40.213799 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f49f3-b915-4f27-8d67-c952a6e6fa2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:40.214418 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f49f3-b915-4f27-8d67-c952a6e6fa2e admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:40.214700 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f49f3-b915-4f27-8d67-c952a6e6fa2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:40.214985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f49f3-b915-4f27-8d67-c952a6e6fa2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:40.215675 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f49f3-b915-4f27-8d67-c952a6e6fa2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:40.217313 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f49f3-b915-4f27-8d67-c952a6e6fa2e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:40.228105 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1405/2834] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:53:40 2022] GET /identity/v3/roles => generated 1427 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:53:40.232432 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4f2ac-4d69-4314-a62b-b3f555f2b606 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:40.244033 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4f2ac-4d69-4314-a62b-b3f555f2b606 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:40.248886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebfd1b-15c6-41b4-be4f-902e80fc3387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:40.249596 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebfd1b-15c6-41b4-be4f-902e80fc3387 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:40.249881 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebfd1b-15c6-41b4-be4f-902e80fc3387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:40.250160 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebfd1b-15c6-41b4-be4f-902e80fc3387 admin admin] PATH_INFO: `/v3/projects/695d099ebda84ddeb89b40a81ba00114/users/400345afd2fc4969937164c442b3d550/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:40.252169 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebfd1b-15c6-41b4-be4f-902e80fc3387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=695d099ebda84ddeb89b40a81ba00114, user_id=400345afd2fc4969937164c442b3d550, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:40.255345 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebfd1b-15c6-41b4-be4f-902e80fc3387 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:40.266465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1430/2835] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:53:40 2022] PUT /identity/v3/projects/695d099ebda84ddeb89b40a81ba00114/users/400345afd2fc4969937164c442b3d550/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:40.277923 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3da4e-ab88-4675-8924-b4dd90dacaa8 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:40.278219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3da4e-ab88-4675-8924-b4dd90dacaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:40.278500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3da4e-ab88-4675-8924-b4dd90dacaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:40.308660 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-93e3da4e-ab88-4675-8924-b4dd90dacaa8 None None] MFA Rules not processed for user `400345afd2fc4969937164c442b3d550`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:40.309919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-93e3da4e-ab88-4675-8924-b4dd90dacaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:40.403270 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1406/2836] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:40 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:53:40.415558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebfd1b-15c6-41b4-be4f-902e80fc3387 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:40.417455 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f7ebfd1b-15c6-41b4-be4f-902e80fc3387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:40.477008 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebfd1b-15c6-41b4-be4f-902e80fc3387 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:40.480700 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910d87f-7fbf-4ca5-9def-6c135200fb4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:40.481376 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910d87f-7fbf-4ca5-9def-6c135200fb4e service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:40.481659 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910d87f-7fbf-4ca5-9def-6c135200fb4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:40.481934 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910d87f-7fbf-4ca5-9def-6c135200fb4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:40.491896 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910d87f-7fbf-4ca5-9def-6c135200fb4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:40.493413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910d87f-7fbf-4ca5-9def-6c135200fb4e service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:40.503627 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1431/2837] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:53:40 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:41.068124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3da4e-ab88-4675-8924-b4dd90dacaa8 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:41.069576 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-93e3da4e-ab88-4675-8924-b4dd90dacaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:41.081494 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3da4e-ab88-4675-8924-b4dd90dacaa8 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:41.086248 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ffa88-05a3-44da-9b11-a68326d8132f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:41.086869 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ffa88-05a3-44da-9b11-a68326d8132f service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:41.087156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ffa88-05a3-44da-9b11-a68326d8132f service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:41.087434 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ffa88-05a3-44da-9b11-a68326d8132f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:41.152219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ffa88-05a3-44da-9b11-a68326d8132f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:41.153730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ffa88-05a3-44da-9b11-a68326d8132f service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:41.164603 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1407/2838] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:53:41 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:42.764415 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910d87f-7fbf-4ca5-9def-6c135200fb4e service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:42.765772 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-2910d87f-7fbf-4ca5-9def-6c135200fb4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:42.825413 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910d87f-7fbf-4ca5-9def-6c135200fb4e service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:42.829029 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35564d-016c-4c72-85ce-07a3b386461e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:42.829681 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35564d-016c-4c72-85ce-07a3b386461e service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:42.829965 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35564d-016c-4c72-85ce-07a3b386461e service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:42.830241 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35564d-016c-4c72-85ce-07a3b386461e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:42.832036 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-dd35564d-016c-4c72-85ce-07a3b386461e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:42.881977 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:42.928937 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35564d-016c-4c72-85ce-07a3b386461e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:42.930587 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35564d-016c-4c72-85ce-07a3b386461e service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:42.944945 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1432/2839] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:53:42 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:43.085730 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ffa88-05a3-44da-9b11-a68326d8132f service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:43.096402 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ffa88-05a3-44da-9b11-a68326d8132f service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:43.104334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf519e78-8750-4344-84e6-813bcd374871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:43.105305 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf519e78-8750-4344-84e6-813bcd374871 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:43.105770 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf519e78-8750-4344-84e6-813bcd374871 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:43.106161 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf519e78-8750-4344-84e6-813bcd374871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:43.108891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-cf519e78-8750-4344-84e6-813bcd374871 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:43.117553 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:43.193746 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf519e78-8750-4344-84e6-813bcd374871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:43.195861 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf519e78-8750-4344-84e6-813bcd374871 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:43.208565 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1408/2840] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:53:43 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:45.423092 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35564d-016c-4c72-85ce-07a3b386461e service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:45.433478 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35564d-016c-4c72-85ce-07a3b386461e service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:45.437312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c8083-cbdc-4986-b378-202a29761bd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:45.438006 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c8083-cbdc-4986-b378-202a29761bd2 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:45.438359 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c8083-cbdc-4986-b378-202a29761bd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:45.438729 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c8083-cbdc-4986-b378-202a29761bd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:45.526275 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c8083-cbdc-4986-b378-202a29761bd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:45.528707 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c8083-cbdc-4986-b378-202a29761bd2 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:45.551791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1433/2841] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:53:45 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:48.019698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf519e78-8750-4344-84e6-813bcd374871 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:48.089607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf519e78-8750-4344-84e6-813bcd374871 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:48.094754 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada9515-d99a-418b-9329-07c83c495656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:48.095404 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada9515-d99a-418b-9329-07c83c495656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:48.095686 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada9515-d99a-418b-9329-07c83c495656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:48.095967 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada9515-d99a-418b-9329-07c83c495656 admin admin] PATH_INFO: `/v3/users/c460374f86c84bfcab833b8ef79e1c0c` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:48.097172 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada9515-d99a-418b-9329-07c83c495656 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c460374f86c84bfcab833b8ef79e1c0c)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:48.099087 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada9515-d99a-418b-9329-07c83c495656 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:48.156317 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8ada9515-d99a-418b-9329-07c83c495656 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c460374f86c84bfcab833b8ef79e1c0c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:48.163267 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8ada9515-d99a-418b-9329-07c83c495656 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c460374f86c84bfcab833b8ef79e1c0c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:48.163946 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8ada9515-d99a-418b-9329-07c83c495656 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c460374f86c84bfcab833b8ef79e1c0c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:48.171707 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8ada9515-d99a-418b-9329-07c83c495656 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c460374f86c84bfcab833b8ef79e1c0c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:48.172347 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8ada9515-d99a-418b-9329-07c83c495656 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c460374f86c84bfcab833b8ef79e1c0c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:48.190692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1409/2842] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:53:48 2022] DELETE /identity/v3/users/c460374f86c84bfcab833b8ef79e1c0c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:48.196255 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fb0fc-9c40-46a8-b40b-a3a80fe34f06 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:48.196780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fb0fc-9c40-46a8-b40b-a3a80fe34f06 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:48.196780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fb0fc-9c40-46a8-b40b-a3a80fe34f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:48.217436 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-1d3fb0fc-9c40-46a8-b40b-a3a80fe34f06 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:48.218646 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1d3fb0fc-9c40-46a8-b40b-a3a80fe34f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:48.299888 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1434/2843] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:48.311497 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada9515-d99a-418b-9329-07c83c495656 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:48.312954 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8ada9515-d99a-418b-9329-07c83c495656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:48.344760 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:48.395964 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada9515-d99a-418b-9329-07c83c495656 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:48.400663 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4d46b-c0e4-41d4-b48c-2485eadfac65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:48.401331 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4d46b-c0e4-41d4-b48c-2485eadfac65 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:48.401620 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4d46b-c0e4-41d4-b48c-2485eadfac65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:48.401892 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4d46b-c0e4-41d4-b48c-2485eadfac65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:48.412082 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4d46b-c0e4-41d4-b48c-2485eadfac65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:48.413650 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4d46b-c0e4-41d4-b48c-2485eadfac65 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:48.424339 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1410/2844] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:53:48 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:48.733883 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fb0fc-9c40-46a8-b40b-a3a80fe34f06 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:48.736541 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-1d3fb0fc-9c40-46a8-b40b-a3a80fe34f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:48.752903 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fb0fc-9c40-46a8-b40b-a3a80fe34f06 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:48.758889 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c3025-11ab-4b1b-bb23-73ba8b09f47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:48.759642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c3025-11ab-4b1b-bb23-73ba8b09f47c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:48.760032 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c3025-11ab-4b1b-bb23-73ba8b09f47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:48.760380 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c3025-11ab-4b1b-bb23-73ba8b09f47c admin admin] PATH_INFO: `/v3/projects/f33b3e4bc13647bc9bf0245653f8c277` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:48.761637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c3025-11ab-4b1b-bb23-73ba8b09f47c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f33b3e4bc13647bc9bf0245653f8c277)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:48.763717 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c3025-11ab-4b1b-bb23-73ba8b09f47c admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:48.819441 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-933c3025-11ab-4b1b-bb23-73ba8b09f47c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f33b3e4bc13647bc9bf0245653f8c277', 'request_id': 'req-933c3025-11ab-4b1b-bb23-73ba8b09f47c'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:48.825815 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-933c3025-11ab-4b1b-bb23-73ba8b09f47c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f33b3e4bc13647bc9bf0245653f8c277', 'request_id': 'req-933c3025-11ab-4b1b-bb23-73ba8b09f47c'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:48.834226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-933c3025-11ab-4b1b-bb23-73ba8b09f47c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f33b3e4bc13647bc9bf0245653f8c277', 'request_id': 'req-933c3025-11ab-4b1b-bb23-73ba8b09f47c'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:48.836432 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:48.849062 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-933c3025-11ab-4b1b-bb23-73ba8b09f47c admin admin] The token cache is being invalidate because project f33b3e4bc13647bc9bf0245653f8c277 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:53:48.850278 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-933c3025-11ab-4b1b-bb23-73ba8b09f47c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:48.851014 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-933c3025-11ab-4b1b-bb23-73ba8b09f47c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:48.852701 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1435/2845] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:53:48 2022] DELETE /identity/v3/projects/f33b3e4bc13647bc9bf0245653f8c277 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:48.857239 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4d46b-c0e4-41d4-b48c-2485eadfac65 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:48.874131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4d46b-c0e4-41d4-b48c-2485eadfac65 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:48.952985 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006711c-1812-4247-b730-69cbbcf0dad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:48.953697 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006711c-1812-4247-b730-69cbbcf0dad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:48.953980 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006711c-1812-4247-b730-69cbbcf0dad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:48.954318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006711c-1812-4247-b730-69cbbcf0dad5 admin admin] PATH_INFO: `/v3/users/a72b13c6e66d417aa690a5e7b0d62918` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:48.955502 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006711c-1812-4247-b730-69cbbcf0dad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a72b13c6e66d417aa690a5e7b0d62918)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:48.957525 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006711c-1812-4247-b730-69cbbcf0dad5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:49.022296 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2006711c-1812-4247-b730-69cbbcf0dad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a72b13c6e66d417aa690a5e7b0d62918'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:49.030520 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2006711c-1812-4247-b730-69cbbcf0dad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a72b13c6e66d417aa690a5e7b0d62918'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:49.031283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2006711c-1812-4247-b730-69cbbcf0dad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a72b13c6e66d417aa690a5e7b0d62918'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:49.041235 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2006711c-1812-4247-b730-69cbbcf0dad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a72b13c6e66d417aa690a5e7b0d62918'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:49.042327 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2006711c-1812-4247-b730-69cbbcf0dad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a72b13c6e66d417aa690a5e7b0d62918'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:49.063028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1411/2846] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:53:48 2022] DELETE /identity/v3/users/a72b13c6e66d417aa690a5e7b0d62918 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:53:49.372181 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c3025-11ab-4b1b-bb23-73ba8b09f47c admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:49.373994 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-933c3025-11ab-4b1b-bb23-73ba8b09f47c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:49.393914 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c3025-11ab-4b1b-bb23-73ba8b09f47c admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:49.464118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a047f-3363-4282-a1cb-9948a644bbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:49.464774 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a047f-3363-4282-a1cb-9948a644bbc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:49.465055 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a047f-3363-4282-a1cb-9948a644bbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:49.465703 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a047f-3363-4282-a1cb-9948a644bbc7 admin admin] PATH_INFO: `/v3/projects/e482a6a346724335b4cc8ee14f74b859` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:49.466601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a047f-3363-4282-a1cb-9948a644bbc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e482a6a346724335b4cc8ee14f74b859)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:49.468525 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a047f-3363-4282-a1cb-9948a644bbc7 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:49.541251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ed8a047f-3363-4282-a1cb-9948a644bbc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e482a6a346724335b4cc8ee14f74b859', 'request_id': 'req-ed8a047f-3363-4282-a1cb-9948a644bbc7'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:49.552421 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ed8a047f-3363-4282-a1cb-9948a644bbc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e482a6a346724335b4cc8ee14f74b859', 'request_id': 'req-ed8a047f-3363-4282-a1cb-9948a644bbc7'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:49.567370 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ed8a047f-3363-4282-a1cb-9948a644bbc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e482a6a346724335b4cc8ee14f74b859', 'request_id': 'req-ed8a047f-3363-4282-a1cb-9948a644bbc7'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:49.580048 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ed8a047f-3363-4282-a1cb-9948a644bbc7 admin admin] The token cache is being invalidate because project e482a6a346724335b4cc8ee14f74b859 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:53:49.580347 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ed8a047f-3363-4282-a1cb-9948a644bbc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:49.581025 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-ed8a047f-3363-4282-a1cb-9948a644bbc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:49.582586 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1436/2847] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:53:49 2022] DELETE /identity/v3/projects/e482a6a346724335b4cc8ee14f74b859 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:49.587283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006711c-1812-4247-b730-69cbbcf0dad5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:49.588704 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2006711c-1812-4247-b730-69cbbcf0dad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:49.602623 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006711c-1812-4247-b730-69cbbcf0dad5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:49.645901 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:49.668386 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341f749-a7f4-4e03-9ab3-a99feb00d412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:49.669374 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341f749-a7f4-4e03-9ab3-a99feb00d412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:49.669519 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341f749-a7f4-4e03-9ab3-a99feb00d412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:49.669736 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341f749-a7f4-4e03-9ab3-a99feb00d412 admin admin] PATH_INFO: `/v3/users/400345afd2fc4969937164c442b3d550` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:49.670975 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341f749-a7f4-4e03-9ab3-a99feb00d412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=400345afd2fc4969937164c442b3d550)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:49.672914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341f749-a7f4-4e03-9ab3-a99feb00d412 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:49.726824 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1341f749-a7f4-4e03-9ab3-a99feb00d412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '400345afd2fc4969937164c442b3d550'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:49.733710 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1341f749-a7f4-4e03-9ab3-a99feb00d412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '400345afd2fc4969937164c442b3d550'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:49.734402 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1341f749-a7f4-4e03-9ab3-a99feb00d412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '400345afd2fc4969937164c442b3d550'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:49.743577 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1341f749-a7f4-4e03-9ab3-a99feb00d412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '400345afd2fc4969937164c442b3d550'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:49.744297 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1341f749-a7f4-4e03-9ab3-a99feb00d412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '400345afd2fc4969937164c442b3d550'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:49.764148 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1412/2848] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:53:49 2022] DELETE /identity/v3/users/400345afd2fc4969937164c442b3d550 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:50.054253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a047f-3363-4282-a1cb-9948a644bbc7 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:50.055937 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-ed8a047f-3363-4282-a1cb-9948a644bbc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:50.069813 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a047f-3363-4282-a1cb-9948a644bbc7 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:50.133665 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed7c46-afe2-4a78-afee-44b44a922f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:50.134320 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed7c46-afe2-4a78-afee-44b44a922f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:50.134606 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed7c46-afe2-4a78-afee-44b44a922f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:50.134888 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed7c46-afe2-4a78-afee-44b44a922f44 admin admin] PATH_INFO: `/v3/projects/695d099ebda84ddeb89b40a81ba00114` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:50.136121 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed7c46-afe2-4a78-afee-44b44a922f44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=695d099ebda84ddeb89b40a81ba00114)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:50.138212 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed7c46-afe2-4a78-afee-44b44a922f44 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:50.163795 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:50.195958 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-c2ed7c46-afe2-4a78-afee-44b44a922f44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '695d099ebda84ddeb89b40a81ba00114', 'request_id': 'req-c2ed7c46-afe2-4a78-afee-44b44a922f44'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:50.203387 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-c2ed7c46-afe2-4a78-afee-44b44a922f44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '695d099ebda84ddeb89b40a81ba00114', 'request_id': 'req-c2ed7c46-afe2-4a78-afee-44b44a922f44'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:50.211664 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-c2ed7c46-afe2-4a78-afee-44b44a922f44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '695d099ebda84ddeb89b40a81ba00114', 'request_id': 'req-c2ed7c46-afe2-4a78-afee-44b44a922f44'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:50.230270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-c2ed7c46-afe2-4a78-afee-44b44a922f44 admin admin] The token cache is being invalidate because project 695d099ebda84ddeb89b40a81ba00114 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:53:50.230556 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-c2ed7c46-afe2-4a78-afee-44b44a922f44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:50.231213 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-c2ed7c46-afe2-4a78-afee-44b44a922f44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:50.232749 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1437/2849] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:53:50 2022] DELETE /identity/v3/projects/695d099ebda84ddeb89b40a81ba00114 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:50.239347 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd91cf-ecba-4a71-80d4-7e2268df5603 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:50.239846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd91cf-ecba-4a71-80d4-7e2268df5603 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:50.240015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd91cf-ecba-4a71-80d4-7e2268df5603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:50.264899 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1ebd91cf-ecba-4a71-80d4-7e2268df5603 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:50.266332 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1ebd91cf-ecba-4a71-80d4-7e2268df5603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:50.328482 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1413/2850] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:50.337716 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3a7a9-90fd-4e22-ab26-c61359ef19e9 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:50.337716 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3a7a9-90fd-4e22-ab26-c61359ef19e9 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:50.337716 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3a7a9-90fd-4e22-ab26-c61359ef19e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:50.359270 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-8af3a7a9-90fd-4e22-ab26-c61359ef19e9 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:50.360470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-8af3a7a9-90fd-4e22-ab26-c61359ef19e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:50.370045 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1438/2851] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:50.374361 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd91cf-ecba-4a71-80d4-7e2268df5603 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:50.384747 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd91cf-ecba-4a71-80d4-7e2268df5603 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:50.389357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e589ba-c7de-4bbd-9154-4b5792e9d5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:50.389970 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e589ba-c7de-4bbd-9154-4b5792e9d5eb admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:50.390256 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e589ba-c7de-4bbd-9154-4b5792e9d5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:50.390535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e589ba-c7de-4bbd-9154-4b5792e9d5eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:50.391228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e589ba-c7de-4bbd-9154-4b5792e9d5eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:53:50.391535 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e589ba-c7de-4bbd-9154-4b5792e9d5eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:50.393150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e589ba-c7de-4bbd-9154-4b5792e9d5eb admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:50.404919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1414/2852] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:53:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:53:50.409364 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3a7a9-90fd-4e22-ab26-c61359ef19e9 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:50.419402 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3a7a9-90fd-4e22-ab26-c61359ef19e9 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:50.423986 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782f483-c0d0-435b-8b23-f2df004651a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:50.424589 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782f483-c0d0-435b-8b23-f2df004651a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:50.424866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782f483-c0d0-435b-8b23-f2df004651a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:50.425165 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782f483-c0d0-435b-8b23-f2df004651a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:50.425836 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782f483-c0d0-435b-8b23-f2df004651a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:50.428308 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782f483-c0d0-435b-8b23-f2df004651a6 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:50.456276 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1439/2853] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:53:50 2022] POST /identity/v3/projects => generated 394 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:53:50.460536 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e589ba-c7de-4bbd-9154-4b5792e9d5eb admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:50.470997 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e589ba-c7de-4bbd-9154-4b5792e9d5eb admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:50.517432 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:50.532135 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e50e22-a968-4292-8958-8075813f0228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:50.532755 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e50e22-a968-4292-8958-8075813f0228 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:50.533034 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e50e22-a968-4292-8958-8075813f0228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:50.533338 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e50e22-a968-4292-8958-8075813f0228 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:50.533992 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e50e22-a968-4292-8958-8075813f0228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:50.536375 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e50e22-a968-4292-8958-8075813f0228 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:50.562574 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1415/2854] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:53:50 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:53:50.563739 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782f483-c0d0-435b-8b23-f2df004651a6 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:50.575825 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782f483-c0d0-435b-8b23-f2df004651a6 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:50.580803 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19534ffb-5220-4619-ac48-2ba4ea0eda2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:50.581462 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19534ffb-5220-4619-ac48-2ba4ea0eda2b admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:50.581740 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19534ffb-5220-4619-ac48-2ba4ea0eda2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:50.582015 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19534ffb-5220-4619-ac48-2ba4ea0eda2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:50.582712 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19534ffb-5220-4619-ac48-2ba4ea0eda2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:50.584395 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19534ffb-5220-4619-ac48-2ba4ea0eda2b admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:50.595243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1440/2855] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:53:50 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:53:50.599386 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e50e22-a968-4292-8958-8075813f0228 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:50.609969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e50e22-a968-4292-8958-8075813f0228 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:50.614560 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8955806-bb9a-4d25-8582-208142110398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:50.615169 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8955806-bb9a-4d25-8582-208142110398 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:50.615978 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8955806-bb9a-4d25-8582-208142110398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:50.616264 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8955806-bb9a-4d25-8582-208142110398 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:50.617131 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8955806-bb9a-4d25-8582-208142110398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:50.618769 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8955806-bb9a-4d25-8582-208142110398 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:50.629665 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1416/2856] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:53:50 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:53:50.634110 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19534ffb-5220-4619-ac48-2ba4ea0eda2b admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:50.644251 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19534ffb-5220-4619-ac48-2ba4ea0eda2b admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:50.649231 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6b8ab-a455-40cb-94a6-d559496d4708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:50.649895 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6b8ab-a455-40cb-94a6-d559496d4708 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:50.650183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6b8ab-a455-40cb-94a6-d559496d4708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:50.650463 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6b8ab-a455-40cb-94a6-d559496d4708 admin admin] PATH_INFO: `/v3/projects/fe68e97e521b43678dd06efe33bb3ec6/users/ba6a5a62934148ccb2f7178ead0a1188/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:50.669272 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6b8ab-a455-40cb-94a6-d559496d4708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe68e97e521b43678dd06efe33bb3ec6, user_id=ba6a5a62934148ccb2f7178ead0a1188, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:50.672113 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6b8ab-a455-40cb-94a6-d559496d4708 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:50.686400 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1441/2857] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:53:50 2022] PUT /identity/v3/projects/fe68e97e521b43678dd06efe33bb3ec6/users/ba6a5a62934148ccb2f7178ead0a1188/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:50.697893 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a0082-ede1-4404-94a1-4061ee5eb346 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:50.698187 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a0082-ede1-4404-94a1-4061ee5eb346 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:50.698467 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a0082-ede1-4404-94a1-4061ee5eb346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:50.732458 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d25a0082-ede1-4404-94a1-4061ee5eb346 None None] MFA Rules not processed for user `ba6a5a62934148ccb2f7178ead0a1188`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:50.733638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d25a0082-ede1-4404-94a1-4061ee5eb346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:50.800631 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1417/2858] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:50 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:53:50.805781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6b8ab-a455-40cb-94a6-d559496d4708 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:50.869594 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6b8ab-a455-40cb-94a6-d559496d4708 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:50.874352 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d3d87-8e02-416b-8184-805be8505cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:50.874967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d3d87-8e02-416b-8184-805be8505cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:50.875253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d3d87-8e02-416b-8184-805be8505cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:50.875531 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d3d87-8e02-416b-8184-805be8505cd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:50.876190 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d3d87-8e02-416b-8184-805be8505cd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:50.878411 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d3d87-8e02-416b-8184-805be8505cd5 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:50.908033 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1442/2859] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:53:50 2022] POST /identity/v3/projects => generated 392 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:53:50.913057 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a0082-ede1-4404-94a1-4061ee5eb346 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:50.924061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a0082-ede1-4404-94a1-4061ee5eb346 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:50.987259 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e6b05-0a41-46cc-9fc9-5d9759213179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:50.987259 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e6b05-0a41-46cc-9fc9-5d9759213179 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:50.987259 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e6b05-0a41-46cc-9fc9-5d9759213179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:50.987638 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e6b05-0a41-46cc-9fc9-5d9759213179 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:50.987994 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e6b05-0a41-46cc-9fc9-5d9759213179 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:50.989956 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e6b05-0a41-46cc-9fc9-5d9759213179 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:51.012777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1418/2860] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:53:50 2022] POST /identity/v3/users => generated 303 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:53:51.017416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d3d87-8e02-416b-8184-805be8505cd5 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:51.028863 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d3d87-8e02-416b-8184-805be8505cd5 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:51.034797 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c976da-fb74-4078-93c9-975070b485e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:51.035962 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c976da-fb74-4078-93c9-975070b485e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:51.035962 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c976da-fb74-4078-93c9-975070b485e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:51.036178 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c976da-fb74-4078-93c9-975070b485e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:51.036875 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c976da-fb74-4078-93c9-975070b485e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:51.039686 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c976da-fb74-4078-93c9-975070b485e4 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:51.052514 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1443/2861] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:53:51 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:53:51.057047 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e6b05-0a41-46cc-9fc9-5d9759213179 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:51.067584 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e6b05-0a41-46cc-9fc9-5d9759213179 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:51.073507 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291e828-1e32-4372-a151-1e7ca303125c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:51.074271 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291e828-1e32-4372-a151-1e7ca303125c admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:51.074559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291e828-1e32-4372-a151-1e7ca303125c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:51.074840 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291e828-1e32-4372-a151-1e7ca303125c admin admin] PATH_INFO: `/v3/projects/6f4b0e1d7d394b9ab13d84c50955f9d3/users/ce8053c71191413db67b558b2e879bf3/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:51.091416 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291e828-1e32-4372-a151-1e7ca303125c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f4b0e1d7d394b9ab13d84c50955f9d3, user_id=ce8053c71191413db67b558b2e879bf3, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:51.094124 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291e828-1e32-4372-a151-1e7ca303125c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:51.108347 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1419/2862] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:53:51 2022] PUT /identity/v3/projects/6f4b0e1d7d394b9ab13d84c50955f9d3/users/ce8053c71191413db67b558b2e879bf3/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:51.112746 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c976da-fb74-4078-93c9-975070b485e4 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:51.122988 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c976da-fb74-4078-93c9-975070b485e4 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:51.158742 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:51.195823 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf2a4b-7518-40ec-9094-224a0e14dece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:51.196451 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf2a4b-7518-40ec-9094-224a0e14dece admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:51.196736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf2a4b-7518-40ec-9094-224a0e14dece admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:51.197014 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf2a4b-7518-40ec-9094-224a0e14dece admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:51.197750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf2a4b-7518-40ec-9094-224a0e14dece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:51.199359 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf2a4b-7518-40ec-9094-224a0e14dece admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:51.209974 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1444/2863] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:53:51 2022] GET /identity/v3/roles => generated 1427 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:53:51.214321 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291e828-1e32-4372-a151-1e7ca303125c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:51.224888 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291e828-1e32-4372-a151-1e7ca303125c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:51.229581 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5347e42-48b1-40d1-a996-0abe841ba917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:51.230269 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5347e42-48b1-40d1-a996-0abe841ba917 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:51.230530 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5347e42-48b1-40d1-a996-0abe841ba917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:51.230843 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5347e42-48b1-40d1-a996-0abe841ba917 admin admin] PATH_INFO: `/v3/projects/6f4b0e1d7d394b9ab13d84c50955f9d3/users/ce8053c71191413db67b558b2e879bf3/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:51.232781 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5347e42-48b1-40d1-a996-0abe841ba917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f4b0e1d7d394b9ab13d84c50955f9d3, user_id=ce8053c71191413db67b558b2e879bf3, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:51.235447 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5347e42-48b1-40d1-a996-0abe841ba917 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:51.245925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1420/2864] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:53:51 2022] PUT /identity/v3/projects/6f4b0e1d7d394b9ab13d84c50955f9d3/users/ce8053c71191413db67b558b2e879bf3/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:51.257888 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdfa37-2cd8-47b3-9e0e-586ab037ee2c None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:51.258183 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdfa37-2cd8-47b3-9e0e-586ab037ee2c None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:51.258470 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdfa37-2cd8-47b3-9e0e-586ab037ee2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:51.287466 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-e2fdfa37-2cd8-47b3-9e0e-586ab037ee2c None None] MFA Rules not processed for user `ce8053c71191413db67b558b2e879bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:51.288962 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e2fdfa37-2cd8-47b3-9e0e-586ab037ee2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:51.362757 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1445/2865] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:51 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:51.374336 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5347e42-48b1-40d1-a996-0abe841ba917 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:51.376058 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a5347e42-48b1-40d1-a996-0abe841ba917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:51.388910 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:51.445196 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5347e42-48b1-40d1-a996-0abe841ba917 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:51.449767 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bd342-5fa2-43e9-bf77-c4252244cb82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:51.450401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bd342-5fa2-43e9-bf77-c4252244cb82 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:51.450686 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bd342-5fa2-43e9-bf77-c4252244cb82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:51.450962 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bd342-5fa2-43e9-bf77-c4252244cb82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:51.463873 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7bd342-5fa2-43e9-bf77-c4252244cb82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:51.465333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7bd342-5fa2-43e9-bf77-c4252244cb82 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:51.476179 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1421/2866] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:53:51 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:51.524669 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdfa37-2cd8-47b3-9e0e-586ab037ee2c None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:51.526108 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e2fdfa37-2cd8-47b3-9e0e-586ab037ee2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:51.538404 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdfa37-2cd8-47b3-9e0e-586ab037ee2c None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:51.542080 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30277a-74b1-4f3f-aaef-c0b3d729065a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:51.542688 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30277a-74b1-4f3f-aaef-c0b3d729065a service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:51.542973 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30277a-74b1-4f3f-aaef-c0b3d729065a service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:51.543254 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30277a-74b1-4f3f-aaef-c0b3d729065a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:51.601613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d30277a-74b1-4f3f-aaef-c0b3d729065a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:51.603060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d30277a-74b1-4f3f-aaef-c0b3d729065a service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:51.612216 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1446/2867] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:53:51 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:55.054855 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bd342-5fa2-43e9-bf77-c4252244cb82 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:55.148310 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bd342-5fa2-43e9-bf77-c4252244cb82 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:55.153276 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9920928-69b3-4f3f-b9a3-ad4d43036a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:55.153959 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9920928-69b3-4f3f-b9a3-ad4d43036a59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:55.154233 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9920928-69b3-4f3f-b9a3-ad4d43036a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:55.155372 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9920928-69b3-4f3f-b9a3-ad4d43036a59 admin admin] PATH_INFO: `/v3/users/ba6a5a62934148ccb2f7178ead0a1188` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:55.156973 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9920928-69b3-4f3f-b9a3-ad4d43036a59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba6a5a62934148ccb2f7178ead0a1188)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:55.159843 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9920928-69b3-4f3f-b9a3-ad4d43036a59 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:55.225091 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b9920928-69b3-4f3f-b9a3-ad4d43036a59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba6a5a62934148ccb2f7178ead0a1188'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:55.233313 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b9920928-69b3-4f3f-b9a3-ad4d43036a59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba6a5a62934148ccb2f7178ead0a1188'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:55.234188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b9920928-69b3-4f3f-b9a3-ad4d43036a59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba6a5a62934148ccb2f7178ead0a1188'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:55.243698 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b9920928-69b3-4f3f-b9a3-ad4d43036a59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba6a5a62934148ccb2f7178ead0a1188'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:55.244514 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b9920928-69b3-4f3f-b9a3-ad4d43036a59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba6a5a62934148ccb2f7178ead0a1188'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:55.267901 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1422/2868] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:53:55 2022] DELETE /identity/v3/users/ba6a5a62934148ccb2f7178ead0a1188 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:53:55.274410 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f2f9f-46d2-4d0b-a794-20596441ad89 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:55.274944 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f2f9f-46d2-4d0b-a794-20596441ad89 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:55.275242 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f2f9f-46d2-4d0b-a794-20596441ad89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:55.298505 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-d81f2f9f-46d2-4d0b-a794-20596441ad89 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:55.299729 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d81f2f9f-46d2-4d0b-a794-20596441ad89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:55.341856 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:55.371343 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1447/2869] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:55.384016 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9920928-69b3-4f3f-b9a3-ad4d43036a59 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:55.385552 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b9920928-69b3-4f3f-b9a3-ad4d43036a59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:55.427029 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:55.459765 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9920928-69b3-4f3f-b9a3-ad4d43036a59 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:55.463874 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca91bf-50f0-4a19-872b-29e45c802952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:55.464582 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca91bf-50f0-4a19-872b-29e45c802952 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:55.464872 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca91bf-50f0-4a19-872b-29e45c802952 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:55.465174 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca91bf-50f0-4a19-872b-29e45c802952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:55.480335 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca91bf-50f0-4a19-872b-29e45c802952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:55.481846 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca91bf-50f0-4a19-872b-29e45c802952 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:55.492867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1423/2870] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:53:55 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:53:55.800964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f2f9f-46d2-4d0b-a794-20596441ad89 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:55.803389 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d81f2f9f-46d2-4d0b-a794-20596441ad89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:55.816893 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f2f9f-46d2-4d0b-a794-20596441ad89 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:55.825229 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee4a95-c2e0-4f32-9bed-cfe293c30bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:55.825879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee4a95-c2e0-4f32-9bed-cfe293c30bbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:55.826158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee4a95-c2e0-4f32-9bed-cfe293c30bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:55.826431 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee4a95-c2e0-4f32-9bed-cfe293c30bbf admin admin] PATH_INFO: `/v3/projects/fe68e97e521b43678dd06efe33bb3ec6` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:55.827621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee4a95-c2e0-4f32-9bed-cfe293c30bbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe68e97e521b43678dd06efe33bb3ec6)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:55.829590 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee4a95-c2e0-4f32-9bed-cfe293c30bbf admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:55.890457 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d4ee4a95-c2e0-4f32-9bed-cfe293c30bbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe68e97e521b43678dd06efe33bb3ec6', 'request_id': 'req-d4ee4a95-c2e0-4f32-9bed-cfe293c30bbf'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:55.898698 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d4ee4a95-c2e0-4f32-9bed-cfe293c30bbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe68e97e521b43678dd06efe33bb3ec6', 'request_id': 'req-d4ee4a95-c2e0-4f32-9bed-cfe293c30bbf'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:55.906982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d4ee4a95-c2e0-4f32-9bed-cfe293c30bbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe68e97e521b43678dd06efe33bb3ec6', 'request_id': 'req-d4ee4a95-c2e0-4f32-9bed-cfe293c30bbf'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:55.919013 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d4ee4a95-c2e0-4f32-9bed-cfe293c30bbf admin admin] The token cache is being invalidate because project fe68e97e521b43678dd06efe33bb3ec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:53:55.919293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d4ee4a95-c2e0-4f32-9bed-cfe293c30bbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:55.919997 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d4ee4a95-c2e0-4f32-9bed-cfe293c30bbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:55.921848 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1448/2871] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:53:55 2022] DELETE /identity/v3/projects/fe68e97e521b43678dd06efe33bb3ec6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:55.925901 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca91bf-50f0-4a19-872b-29e45c802952 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:55.936544 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca91bf-50f0-4a19-872b-29e45c802952 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:56.009462 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef7390-d97b-4198-b6d6-5038732e33ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:56.010098 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef7390-d97b-4198-b6d6-5038732e33ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:56.010364 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef7390-d97b-4198-b6d6-5038732e33ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:56.010647 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef7390-d97b-4198-b6d6-5038732e33ae admin admin] PATH_INFO: `/v3/users/ce8053c71191413db67b558b2e879bf3` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:56.011800 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef7390-d97b-4198-b6d6-5038732e33ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce8053c71191413db67b558b2e879bf3)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:56.013752 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef7390-d97b-4198-b6d6-5038732e33ae admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:56.064278 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-aeef7390-d97b-4198-b6d6-5038732e33ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce8053c71191413db67b558b2e879bf3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:56.073773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-aeef7390-d97b-4198-b6d6-5038732e33ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce8053c71191413db67b558b2e879bf3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:56.074541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-aeef7390-d97b-4198-b6d6-5038732e33ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce8053c71191413db67b558b2e879bf3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:56.082963 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-aeef7390-d97b-4198-b6d6-5038732e33ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce8053c71191413db67b558b2e879bf3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:56.084002 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-aeef7390-d97b-4198-b6d6-5038732e33ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce8053c71191413db67b558b2e879bf3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:56.104188 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1424/2872] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:53:55 2022] DELETE /identity/v3/users/ce8053c71191413db67b558b2e879bf3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:53:56.396879 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee4a95-c2e0-4f32-9bed-cfe293c30bbf admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:56.398409 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d4ee4a95-c2e0-4f32-9bed-cfe293c30bbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:56.414333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee4a95-c2e0-4f32-9bed-cfe293c30bbf admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:56.482117 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92318596-a861-432e-af02-47e9b0574ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:56.482750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92318596-a861-432e-af02-47e9b0574ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:56.483030 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92318596-a861-432e-af02-47e9b0574ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:56.483302 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92318596-a861-432e-af02-47e9b0574ccf admin admin] PATH_INFO: `/v3/projects/6f4b0e1d7d394b9ab13d84c50955f9d3` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:56.484461 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92318596-a861-432e-af02-47e9b0574ccf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f4b0e1d7d394b9ab13d84c50955f9d3)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:56.486516 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92318596-a861-432e-af02-47e9b0574ccf admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:56.534249 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-92318596-a861-432e-af02-47e9b0574ccf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f4b0e1d7d394b9ab13d84c50955f9d3', 'request_id': 'req-92318596-a861-432e-af02-47e9b0574ccf'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:56.541080 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-92318596-a861-432e-af02-47e9b0574ccf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f4b0e1d7d394b9ab13d84c50955f9d3', 'request_id': 'req-92318596-a861-432e-af02-47e9b0574ccf'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:56.550613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-92318596-a861-432e-af02-47e9b0574ccf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f4b0e1d7d394b9ab13d84c50955f9d3', 'request_id': 'req-92318596-a861-432e-af02-47e9b0574ccf'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:56.556562 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:56.564390 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-92318596-a861-432e-af02-47e9b0574ccf admin admin] The token cache is being invalidate because project 6f4b0e1d7d394b9ab13d84c50955f9d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:53:56.564904 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-92318596-a861-432e-af02-47e9b0574ccf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:56.565708 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-92318596-a861-432e-af02-47e9b0574ccf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:53:56.567850 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1449/2873] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:53:56 2022] DELETE /identity/v3/projects/6f4b0e1d7d394b9ab13d84c50955f9d3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:56.574083 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb8246-db07-4635-9135-ae99b1cc0465 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:56.575142 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb8246-db07-4635-9135-ae99b1cc0465 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:56.575142 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb8246-db07-4635-9135-ae99b1cc0465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:56.598089 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-bcbb8246-db07-4635-9135-ae99b1cc0465 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:56.599787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bcbb8246-db07-4635-9135-ae99b1cc0465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:56.648674 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:56.666609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1425/2874] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:56.675484 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2242c-1ec8-43de-9112-227efd372e0b None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:56.675830 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2242c-1ec8-43de-9112-227efd372e0b None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:56.676153 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2242c-1ec8-43de-9112-227efd372e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:56.697161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-99f2242c-1ec8-43de-9112-227efd372e0b None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:56.698375 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-99f2242c-1ec8-43de-9112-227efd372e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:56.706317 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1450/2875] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:56.711814 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb8246-db07-4635-9135-ae99b1cc0465 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:56.722349 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb8246-db07-4635-9135-ae99b1cc0465 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:56.727680 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a76ec6-745c-4729-b8fc-4e686f80f996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:56.728298 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a76ec6-745c-4729-b8fc-4e686f80f996 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:56.728579 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a76ec6-745c-4729-b8fc-4e686f80f996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:56.728856 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a76ec6-745c-4729-b8fc-4e686f80f996 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:56.729583 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a76ec6-745c-4729-b8fc-4e686f80f996 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:53:56.729891 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a76ec6-745c-4729-b8fc-4e686f80f996 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:56.731486 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a76ec6-745c-4729-b8fc-4e686f80f996 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:56.753216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1426/2876] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:53:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:53:56.759200 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f2242c-1ec8-43de-9112-227efd372e0b None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:56.777161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f2242c-1ec8-43de-9112-227efd372e0b None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:56.782180 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58432f-c6e8-4745-a49b-d63a1dd98243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:56.782815 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58432f-c6e8-4745-a49b-d63a1dd98243 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:56.783093 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58432f-c6e8-4745-a49b-d63a1dd98243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:56.783364 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58432f-c6e8-4745-a49b-d63a1dd98243 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:56.784031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58432f-c6e8-4745-a49b-d63a1dd98243 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:56.786083 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58432f-c6e8-4745-a49b-d63a1dd98243 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:56.821725 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1451/2877] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:53:56 2022] POST /identity/v3/projects => generated 410 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:53:56.828647 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a76ec6-745c-4729-b8fc-4e686f80f996 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:56.840344 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a76ec6-745c-4729-b8fc-4e686f80f996 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:56.917529 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7b9c8-04fc-413d-abe7-273a63e0a83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:56.918163 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7b9c8-04fc-413d-abe7-273a63e0a83a admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:56.918444 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7b9c8-04fc-413d-abe7-273a63e0a83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:56.918725 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7b9c8-04fc-413d-abe7-273a63e0a83a admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:56.919400 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7b9c8-04fc-413d-abe7-273a63e0a83a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:56.921475 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7b9c8-04fc-413d-abe7-273a63e0a83a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:56.947501 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1427/2878] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:53:56 2022] POST /identity/v3/users => generated 306 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:53:56.951983 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58432f-c6e8-4745-a49b-d63a1dd98243 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:56.963734 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58432f-c6e8-4745-a49b-d63a1dd98243 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:56.968687 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0d196-1779-4f72-82f8-44becfa62561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:56.969329 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0d196-1779-4f72-82f8-44becfa62561 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:56.969608 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0d196-1779-4f72-82f8-44becfa62561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:56.969883 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0d196-1779-4f72-82f8-44becfa62561 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:56.970584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0d196-1779-4f72-82f8-44becfa62561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:56.972220 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0d196-1779-4f72-82f8-44becfa62561 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:56.983520 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1452/2879] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:53:56 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:53:56.987801 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7b9c8-04fc-413d-abe7-273a63e0a83a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:57.001026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7b9c8-04fc-413d-abe7-273a63e0a83a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:57.006344 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1cd049-f9c6-4e19-a5c0-00a13635d43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:57.007038 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cd049-f9c6-4e19-a5c0-00a13635d43c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:57.007338 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cd049-f9c6-4e19-a5c0-00a13635d43c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:57.007640 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cd049-f9c6-4e19-a5c0-00a13635d43c admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:57.008385 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1cd049-f9c6-4e19-a5c0-00a13635d43c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:57.010177 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1cd049-f9c6-4e19-a5c0-00a13635d43c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:57.023557 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1428/2880] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:53:56 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:53:57.028715 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0d196-1779-4f72-82f8-44becfa62561 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:57.040790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0d196-1779-4f72-82f8-44becfa62561 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:57.047668 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9b688-62db-4144-9212-8b920b6684b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:57.048728 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9b688-62db-4144-9212-8b920b6684b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:57.049214 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9b688-62db-4144-9212-8b920b6684b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:57.049556 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9b688-62db-4144-9212-8b920b6684b3 admin admin] PATH_INFO: `/v3/projects/0ebf201c68e44257baeec6c14b7bfa5a/users/cdec27e5081c43f989150c956b399b39/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:57.072483 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be9b688-62db-4144-9212-8b920b6684b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ebf201c68e44257baeec6c14b7bfa5a, user_id=cdec27e5081c43f989150c956b399b39, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:57.075208 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be9b688-62db-4144-9212-8b920b6684b3 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:57.090277 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1453/2881] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:53:57 2022] PUT /identity/v3/projects/0ebf201c68e44257baeec6c14b7bfa5a/users/cdec27e5081c43f989150c956b399b39/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:57.102867 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cf64b-9f28-4380-8d43-8a76da0affd4 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:57.103422 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cf64b-9f28-4380-8d43-8a76da0affd4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:57.103422 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cf64b-9f28-4380-8d43-8a76da0affd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:57.137803 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-dc8cf64b-9f28-4380-8d43-8a76da0affd4 None None] MFA Rules not processed for user `cdec27e5081c43f989150c956b399b39`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:57.139272 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-dc8cf64b-9f28-4380-8d43-8a76da0affd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:57.209539 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1429/2882] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:57 2022] POST /identity/v3/auth/tokens => generated 3296 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:53:57.218458 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9b688-62db-4144-9212-8b920b6684b3 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:57.282444 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9b688-62db-4144-9212-8b920b6684b3 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:57.288243 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdd22a-846f-44ac-a547-20935db295a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:57.288871 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdd22a-846f-44ac-a547-20935db295a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:57.289186 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdd22a-846f-44ac-a547-20935db295a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:57.289733 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdd22a-846f-44ac-a547-20935db295a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:57.290714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bdd22a-846f-44ac-a547-20935db295a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:57.294253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bdd22a-846f-44ac-a547-20935db295a6 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:57.333087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1454/2883] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:53:57 2022] POST /identity/v3/projects => generated 410 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:53:57.338439 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8cf64b-9f28-4380-8d43-8a76da0affd4 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:57.357548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8cf64b-9f28-4380-8d43-8a76da0affd4 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:57.416148 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:57.432777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b805319-34e5-4d91-b5d8-821fe226408f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:57.433624 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b805319-34e5-4d91-b5d8-821fe226408f admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:57.433913 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b805319-34e5-4d91-b5d8-821fe226408f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:57.434193 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b805319-34e5-4d91-b5d8-821fe226408f admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:57.434873 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b805319-34e5-4d91-b5d8-821fe226408f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:57.436792 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b805319-34e5-4d91-b5d8-821fe226408f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:57.467328 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1430/2884] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:53:57 2022] POST /identity/v3/users => generated 312 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:53:57.473137 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdd22a-846f-44ac-a547-20935db295a6 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:57.484595 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdd22a-846f-44ac-a547-20935db295a6 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:57.489588 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c349c6-73fc-4545-ba52-d038330e83b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:57.490815 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c349c6-73fc-4545-ba52-d038330e83b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:57.491454 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c349c6-73fc-4545-ba52-d038330e83b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:57.491454 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c349c6-73fc-4545-ba52-d038330e83b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:57.492031 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c349c6-73fc-4545-ba52-d038330e83b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:57.493685 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c349c6-73fc-4545-ba52-d038330e83b6 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:57.509476 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1455/2885] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:53:57 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:53:57.515686 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b805319-34e5-4d91-b5d8-821fe226408f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:57.530211 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b805319-34e5-4d91-b5d8-821fe226408f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:57.535334 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e06aa-3089-42bf-a8d5-75de12d82bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:57.536032 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e06aa-3089-42bf-a8d5-75de12d82bcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:57.536323 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e06aa-3089-42bf-a8d5-75de12d82bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:57.536601 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e06aa-3089-42bf-a8d5-75de12d82bcf admin admin] PATH_INFO: `/v3/projects/27cd6e892f284f2db0b584cc633429bd/users/5d24ca33d61f48dfa7b334bd1c780eb3/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:57.555604 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e06aa-3089-42bf-a8d5-75de12d82bcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=27cd6e892f284f2db0b584cc633429bd, user_id=5d24ca33d61f48dfa7b334bd1c780eb3, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:57.558278 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e06aa-3089-42bf-a8d5-75de12d82bcf admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:57.576235 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1431/2886] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:53:57 2022] PUT /identity/v3/projects/27cd6e892f284f2db0b584cc633429bd/users/5d24ca33d61f48dfa7b334bd1c780eb3/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:57.581079 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c349c6-73fc-4545-ba52-d038330e83b6 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:57.591758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c349c6-73fc-4545-ba52-d038330e83b6 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:57.660858 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a03c34-f88a-44fc-86c5-96c80d9b5277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:57.661637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a03c34-f88a-44fc-86c5-96c80d9b5277 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:57.661982 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a03c34-f88a-44fc-86c5-96c80d9b5277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:57.662328 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a03c34-f88a-44fc-86c5-96c80d9b5277 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:57.663185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a03c34-f88a-44fc-86c5-96c80d9b5277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:57.664980 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a03c34-f88a-44fc-86c5-96c80d9b5277 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:57.676778 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1456/2887] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:53:57 2022] GET /identity/v3/roles => generated 1427 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:53:57.682009 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e06aa-3089-42bf-a8d5-75de12d82bcf admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:57.693918 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e06aa-3089-42bf-a8d5-75de12d82bcf admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:57.698859 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a446e7-9e18-4c7a-89f4-69b96155df8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:57.699577 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a446e7-9e18-4c7a-89f4-69b96155df8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:57.699862 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a446e7-9e18-4c7a-89f4-69b96155df8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:57.700155 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a446e7-9e18-4c7a-89f4-69b96155df8c admin admin] PATH_INFO: `/v3/projects/27cd6e892f284f2db0b584cc633429bd/users/5d24ca33d61f48dfa7b334bd1c780eb3/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:57.702257 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a446e7-9e18-4c7a-89f4-69b96155df8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=27cd6e892f284f2db0b584cc633429bd, user_id=5d24ca33d61f48dfa7b334bd1c780eb3, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:57.705096 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a446e7-9e18-4c7a-89f4-69b96155df8c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:57.718289 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1432/2888] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:53:57 2022] PUT /identity/v3/projects/27cd6e892f284f2db0b584cc633429bd/users/5d24ca33d61f48dfa7b334bd1c780eb3/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:53:57.729729 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96502d6-5f14-40ec-b822-6450bac0f7c7 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:57.730022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96502d6-5f14-40ec-b822-6450bac0f7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:57.730307 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96502d6-5f14-40ec-b822-6450bac0f7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:57.737792 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:57.768799 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-c96502d6-5f14-40ec-b822-6450bac0f7c7 None None] MFA Rules not processed for user `5d24ca33d61f48dfa7b334bd1c780eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:57.770303 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c96502d6-5f14-40ec-b822-6450bac0f7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:57.845964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1457/2889] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:53:57 2022] POST /identity/v3/auth/tokens => generated 3363 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:57.858360 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a446e7-9e18-4c7a-89f4-69b96155df8c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:57.860102 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-64a446e7-9e18-4c7a-89f4-69b96155df8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:57.923499 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a446e7-9e18-4c7a-89f4-69b96155df8c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:57.928151 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e7276-e7e0-4c0e-8235-268f891e5fcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:57.929176 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e7276-e7e0-4c0e-8235-268f891e5fcd service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:57.929468 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e7276-e7e0-4c0e-8235-268f891e5fcd service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:57.929773 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e7276-e7e0-4c0e-8235-268f891e5fcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:57.939721 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e7276-e7e0-4c0e-8235-268f891e5fcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:57.941203 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e7276-e7e0-4c0e-8235-268f891e5fcd service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:57.951887 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1433/2890] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:53:57 2022] GET /identity/v3/auth/tokens => generated 3363 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:58.150481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96502d6-5f14-40ec-b822-6450bac0f7c7 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:58.160758 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96502d6-5f14-40ec-b822-6450bac0f7c7 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:58.165832 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80f18d-aee3-4969-ab64-7553ddaf38fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:58.166462 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80f18d-aee3-4969-ab64-7553ddaf38fe service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:58.166750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80f18d-aee3-4969-ab64-7553ddaf38fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:58.167039 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80f18d-aee3-4969-ab64-7553ddaf38fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:58.239209 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80f18d-aee3-4969-ab64-7553ddaf38fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:58.241004 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80f18d-aee3-4969-ab64-7553ddaf38fe service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:58.252862 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1458/2891] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:53:58 2022] GET /identity/v3/auth/tokens => generated 3296 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:54:06.971816 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e7276-e7e0-4c0e-8235-268f891e5fcd service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:07.087561 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e7276-e7e0-4c0e-8235-268f891e5fcd service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:07.092442 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b812d-8f50-4d09-b495-64ae869a84d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:07.093212 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b812d-8f50-4d09-b495-64ae869a84d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:07.093571 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b812d-8f50-4d09-b495-64ae869a84d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:07.093919 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b812d-8f50-4d09-b495-64ae869a84d5 admin admin] PATH_INFO: `/v3/users/cdec27e5081c43f989150c956b399b39` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:07.095248 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b812d-8f50-4d09-b495-64ae869a84d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdec27e5081c43f989150c956b399b39)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:07.097285 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b812d-8f50-4d09-b495-64ae869a84d5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:07.145498 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:07.170180 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-521b812d-8f50-4d09-b495-64ae869a84d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdec27e5081c43f989150c956b399b39'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:07.178887 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-521b812d-8f50-4d09-b495-64ae869a84d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdec27e5081c43f989150c956b399b39'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:07.179729 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-521b812d-8f50-4d09-b495-64ae869a84d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdec27e5081c43f989150c956b399b39'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:07.189292 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-521b812d-8f50-4d09-b495-64ae869a84d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdec27e5081c43f989150c956b399b39'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:07.190040 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-521b812d-8f50-4d09-b495-64ae869a84d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdec27e5081c43f989150c956b399b39'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:07.211511 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1434/2892] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:54:06 2022] DELETE /identity/v3/users/cdec27e5081c43f989150c956b399b39 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:54:07.216305 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a45ac-f6ae-4cff-abfa-11547177f492 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:07.216775 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a45ac-f6ae-4cff-abfa-11547177f492 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:07.216980 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a45ac-f6ae-4cff-abfa-11547177f492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:07.252797 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-658a45ac-f6ae-4cff-abfa-11547177f492 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:54:07.254060 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-658a45ac-f6ae-4cff-abfa-11547177f492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:07.321281 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1459/2893] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:54:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:54:07.332299 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b812d-8f50-4d09-b495-64ae869a84d5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:07.334739 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-521b812d-8f50-4d09-b495-64ae869a84d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:07.428680 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b812d-8f50-4d09-b495-64ae869a84d5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:07.435092 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744e846-e12c-4c08-96bd-9e42a7038ec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:07.436190 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744e846-e12c-4c08-96bd-9e42a7038ec1 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:07.436692 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744e846-e12c-4c08-96bd-9e42a7038ec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:07.437234 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744e846-e12c-4c08-96bd-9e42a7038ec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:07.454268 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a744e846-e12c-4c08-96bd-9e42a7038ec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:07.455722 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a744e846-e12c-4c08-96bd-9e42a7038ec1 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:07.466967 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1435/2894] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:54:07 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:54:07.797714 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a45ac-f6ae-4cff-abfa-11547177f492 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:07.799622 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-658a45ac-f6ae-4cff-abfa-11547177f492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:07.815258 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a45ac-f6ae-4cff-abfa-11547177f492 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:07.821202 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3305d9-f2f2-4da9-b71f-48c0102178ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:07.821946 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3305d9-f2f2-4da9-b71f-48c0102178ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:07.822204 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3305d9-f2f2-4da9-b71f-48c0102178ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:07.822601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3305d9-f2f2-4da9-b71f-48c0102178ab admin admin] PATH_INFO: `/v3/projects/0ebf201c68e44257baeec6c14b7bfa5a` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:07.823860 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3305d9-f2f2-4da9-b71f-48c0102178ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ebf201c68e44257baeec6c14b7bfa5a)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:07.835786 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3305d9-f2f2-4da9-b71f-48c0102178ab admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:07.854512 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=100 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:07.910754 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3f3305d9-f2f2-4da9-b71f-48c0102178ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ebf201c68e44257baeec6c14b7bfa5a', 'request_id': 'req-3f3305d9-f2f2-4da9-b71f-48c0102178ab'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:07.918911 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3f3305d9-f2f2-4da9-b71f-48c0102178ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ebf201c68e44257baeec6c14b7bfa5a', 'request_id': 'req-3f3305d9-f2f2-4da9-b71f-48c0102178ab'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:07.929170 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3f3305d9-f2f2-4da9-b71f-48c0102178ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ebf201c68e44257baeec6c14b7bfa5a', 'request_id': 'req-3f3305d9-f2f2-4da9-b71f-48c0102178ab'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:07.941594 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3f3305d9-f2f2-4da9-b71f-48c0102178ab admin admin] The token cache is being invalidate because project 0ebf201c68e44257baeec6c14b7bfa5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:54:07.941930 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3f3305d9-f2f2-4da9-b71f-48c0102178ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:07.942704 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-3f3305d9-f2f2-4da9-b71f-48c0102178ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:07.944761 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1460/2895] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:54:07 2022] DELETE /identity/v3/projects/0ebf201c68e44257baeec6c14b7bfa5a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:54:07.948998 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744e846-e12c-4c08-96bd-9e42a7038ec1 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:07.960026 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744e846-e12c-4c08-96bd-9e42a7038ec1 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:08.045925 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f34407-a4c7-42e8-a6ea-c37d7fb89ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:08.046706 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f34407-a4c7-42e8-a6ea-c37d7fb89ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:08.046826 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f34407-a4c7-42e8-a6ea-c37d7fb89ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:08.047032 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f34407-a4c7-42e8-a6ea-c37d7fb89ba9 admin admin] PATH_INFO: `/v3/users/5d24ca33d61f48dfa7b334bd1c780eb3` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:08.048342 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f34407-a4c7-42e8-a6ea-c37d7fb89ba9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d24ca33d61f48dfa7b334bd1c780eb3)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:08.050413 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f34407-a4c7-42e8-a6ea-c37d7fb89ba9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:08.108673 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-32f34407-a4c7-42e8-a6ea-c37d7fb89ba9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d24ca33d61f48dfa7b334bd1c780eb3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:08.110957 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:08.118952 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-32f34407-a4c7-42e8-a6ea-c37d7fb89ba9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d24ca33d61f48dfa7b334bd1c780eb3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:08.119962 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-32f34407-a4c7-42e8-a6ea-c37d7fb89ba9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d24ca33d61f48dfa7b334bd1c780eb3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:08.135532 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-32f34407-a4c7-42e8-a6ea-c37d7fb89ba9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d24ca33d61f48dfa7b334bd1c780eb3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:08.136854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-32f34407-a4c7-42e8-a6ea-c37d7fb89ba9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d24ca33d61f48dfa7b334bd1c780eb3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:08.163072 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1436/2896] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:54:07 2022] DELETE /identity/v3/users/5d24ca33d61f48dfa7b334bd1c780eb3 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:54:08.472621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3305d9-f2f2-4da9-b71f-48c0102178ab admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:08.475918 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-3f3305d9-f2f2-4da9-b71f-48c0102178ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:08.492290 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3305d9-f2f2-4da9-b71f-48c0102178ab admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:08.570794 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d30d0-8494-42e5-a51a-912207037133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:08.571333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d30d0-8494-42e5-a51a-912207037133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:08.571622 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d30d0-8494-42e5-a51a-912207037133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:08.571959 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d30d0-8494-42e5-a51a-912207037133 admin admin] PATH_INFO: `/v3/projects/27cd6e892f284f2db0b584cc633429bd` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:08.573263 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d30d0-8494-42e5-a51a-912207037133 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27cd6e892f284f2db0b584cc633429bd)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:08.575226 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d30d0-8494-42e5-a51a-912207037133 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:08.635299 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8f5d30d0-8494-42e5-a51a-912207037133 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27cd6e892f284f2db0b584cc633429bd', 'request_id': 'req-8f5d30d0-8494-42e5-a51a-912207037133'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:08.642395 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8f5d30d0-8494-42e5-a51a-912207037133 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27cd6e892f284f2db0b584cc633429bd', 'request_id': 'req-8f5d30d0-8494-42e5-a51a-912207037133'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:08.650902 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8f5d30d0-8494-42e5-a51a-912207037133 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27cd6e892f284f2db0b584cc633429bd', 'request_id': 'req-8f5d30d0-8494-42e5-a51a-912207037133'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:08.665235 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8f5d30d0-8494-42e5-a51a-912207037133 admin admin] The token cache is being invalidate because project 27cd6e892f284f2db0b584cc633429bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:54:08.665530 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8f5d30d0-8494-42e5-a51a-912207037133 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:08.666202 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-8f5d30d0-8494-42e5-a51a-912207037133 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:08.667771 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1461/2897] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:54:08 2022] DELETE /identity/v3/projects/27cd6e892f284f2db0b584cc633429bd => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:54:08.673247 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa9fc8-bb37-4691-b536-7b790d32faf8 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:08.673541 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa9fc8-bb37-4691-b536-7b790d32faf8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:08.673819 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa9fc8-bb37-4691-b536-7b790d32faf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:08.696125 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c0fa9fc8-bb37-4691-b536-7b790d32faf8 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:54:08.706430 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c0fa9fc8-bb37-4691-b536-7b790d32faf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:08.770265 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1437/2898] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:54:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:54:08.779791 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0f8e6-bd9d-4f7e-b958-5b9cbe8fbf00 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:08.780071 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0f8e6-bd9d-4f7e-b958-5b9cbe8fbf00 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:08.780343 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0f8e6-bd9d-4f7e-b958-5b9cbe8fbf00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:08.802847 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-e0a0f8e6-bd9d-4f7e-b958-5b9cbe8fbf00 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:54:08.804471 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e0a0f8e6-bd9d-4f7e-b958-5b9cbe8fbf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:08.813678 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1462/2899] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:54:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:54:08.818678 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa9fc8-bb37-4691-b536-7b790d32faf8 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:08.828934 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa9fc8-bb37-4691-b536-7b790d32faf8 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:08.834989 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143825de-189e-4213-a706-b9b2b5c249f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:08.835685 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143825de-189e-4213-a706-b9b2b5c249f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:08.836023 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143825de-189e-4213-a706-b9b2b5c249f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:08.836391 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143825de-189e-4213-a706-b9b2b5c249f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:08.837192 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143825de-189e-4213-a706-b9b2b5c249f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:54:08.837558 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143825de-189e-4213-a706-b9b2b5c249f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:08.839208 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143825de-189e-4213-a706-b9b2b5c249f2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:08.853022 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1438/2900] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:54:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:54:08.859864 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0f8e6-bd9d-4f7e-b958-5b9cbe8fbf00 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:08.875319 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0f8e6-bd9d-4f7e-b958-5b9cbe8fbf00 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:08.881655 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71192878-1d80-462e-8022-d0f178e63944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:08.882282 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71192878-1d80-462e-8022-d0f178e63944 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:08.882565 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71192878-1d80-462e-8022-d0f178e63944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:08.882840 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71192878-1d80-462e-8022-d0f178e63944 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:08.883507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71192878-1d80-462e-8022-d0f178e63944 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:08.885463 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71192878-1d80-462e-8022-d0f178e63944 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:08.918620 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:08.929780 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1463/2901] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:54:08 2022] POST /identity/v3/projects => generated 374 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:54:08.934784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143825de-189e-4213-a706-b9b2b5c249f2 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:08.948204 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143825de-189e-4213-a706-b9b2b5c249f2 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:09.028724 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d13ed-ecaf-4abe-b116-288c174e089a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:09.029542 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d13ed-ecaf-4abe-b116-288c174e089a admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:09.029930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d13ed-ecaf-4abe-b116-288c174e089a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:09.030318 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d13ed-ecaf-4abe-b116-288c174e089a admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:09.031104 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d13ed-ecaf-4abe-b116-288c174e089a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:09.033608 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d13ed-ecaf-4abe-b116-288c174e089a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:09.059927 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1439/2902] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:54:08 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:54:09.064212 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71192878-1d80-462e-8022-d0f178e63944 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:09.076711 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71192878-1d80-462e-8022-d0f178e63944 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:09.081900 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474af2fa-cde3-42fc-a662-1ed5fed92421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:09.082513 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474af2fa-cde3-42fc-a662-1ed5fed92421 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:09.082833 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474af2fa-cde3-42fc-a662-1ed5fed92421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:09.083191 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474af2fa-cde3-42fc-a662-1ed5fed92421 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:09.084058 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474af2fa-cde3-42fc-a662-1ed5fed92421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:09.085925 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474af2fa-cde3-42fc-a662-1ed5fed92421 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:09.098163 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1464/2903] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:54:09 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:54:09.105618 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d13ed-ecaf-4abe-b116-288c174e089a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:09.116806 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d13ed-ecaf-4abe-b116-288c174e089a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:09.122005 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d07e93-71ea-40ea-9a4a-3fdfd2d94a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:09.122616 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d07e93-71ea-40ea-9a4a-3fdfd2d94a17 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:09.122903 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d07e93-71ea-40ea-9a4a-3fdfd2d94a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:09.123187 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d07e93-71ea-40ea-9a4a-3fdfd2d94a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:09.123884 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d07e93-71ea-40ea-9a4a-3fdfd2d94a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:09.125524 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d07e93-71ea-40ea-9a4a-3fdfd2d94a17 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:09.139787 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1440/2904] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:54:09 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:54:09.144642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474af2fa-cde3-42fc-a662-1ed5fed92421 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:09.155477 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474af2fa-cde3-42fc-a662-1ed5fed92421 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:09.163301 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444bb30-93ee-4f21-abda-b6b4098d861d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:09.164437 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444bb30-93ee-4f21-abda-b6b4098d861d admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:09.164979 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444bb30-93ee-4f21-abda-b6b4098d861d admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:09.165278 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444bb30-93ee-4f21-abda-b6b4098d861d admin admin] PATH_INFO: `/v3/projects/c5662a2022f44c51bb9e4b989974caf3/users/5ccbee1febf64fe394089aa40f58380f/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:09.183161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444bb30-93ee-4f21-abda-b6b4098d861d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5662a2022f44c51bb9e4b989974caf3, user_id=5ccbee1febf64fe394089aa40f58380f, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:09.185931 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444bb30-93ee-4f21-abda-b6b4098d861d admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:09.205797 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1465/2905] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:54:09 2022] PUT /identity/v3/projects/c5662a2022f44c51bb9e4b989974caf3/users/5ccbee1febf64fe394089aa40f58380f/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:54:09.226794 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ca826-9e27-47e6-92c8-ceea443246d4 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:09.227282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ca826-9e27-47e6-92c8-ceea443246d4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:09.227780 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ca826-9e27-47e6-92c8-ceea443246d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:09.267401 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-674ca826-9e27-47e6-92c8-ceea443246d4 None None] MFA Rules not processed for user `5ccbee1febf64fe394089aa40f58380f`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:54:09.268879 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-674ca826-9e27-47e6-92c8-ceea443246d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:09.294247 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:09.359683 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1441/2906] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:54:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:54:09.372411 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444bb30-93ee-4f21-abda-b6b4098d861d admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:09.374244 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-c444bb30-93ee-4f21-abda-b6b4098d861d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:09.447085 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444bb30-93ee-4f21-abda-b6b4098d861d admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:09.451506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e469f4-0c5f-4f16-8c64-579d97747f3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:09.452132 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e469f4-0c5f-4f16-8c64-579d97747f3e service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:09.452418 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e469f4-0c5f-4f16-8c64-579d97747f3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:09.452843 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e469f4-0c5f-4f16-8c64-579d97747f3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:09.463975 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e469f4-0c5f-4f16-8c64-579d97747f3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:09.465530 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e469f4-0c5f-4f16-8c64-579d97747f3e service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:09.476774 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1466/2907] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:54:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:54:09.514854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ca826-9e27-47e6-92c8-ceea443246d4 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:09.603882 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ca826-9e27-47e6-92c8-ceea443246d4 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:09.610527 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a7bcd-98d0-43d4-a01f-367b434bf60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:09.611371 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a7bcd-98d0-43d4-a01f-367b434bf60f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:09.612093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a7bcd-98d0-43d4-a01f-367b434bf60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:09.612093 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a7bcd-98d0-43d4-a01f-367b434bf60f admin admin] PATH_INFO: `/v3/users/5ccbee1febf64fe394089aa40f58380f` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:09.613588 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a7bcd-98d0-43d4-a01f-367b434bf60f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ccbee1febf64fe394089aa40f58380f)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:09.615907 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a7bcd-98d0-43d4-a01f-367b434bf60f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:09.703527 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-877a7bcd-98d0-43d4-a01f-367b434bf60f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ccbee1febf64fe394089aa40f58380f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:09.712363 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-877a7bcd-98d0-43d4-a01f-367b434bf60f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ccbee1febf64fe394089aa40f58380f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:09.713097 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-877a7bcd-98d0-43d4-a01f-367b434bf60f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ccbee1febf64fe394089aa40f58380f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:09.722954 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-877a7bcd-98d0-43d4-a01f-367b434bf60f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ccbee1febf64fe394089aa40f58380f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:09.724422 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-877a7bcd-98d0-43d4-a01f-367b434bf60f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ccbee1febf64fe394089aa40f58380f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:09.749997 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1442/2908] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:54:09 2022] DELETE /identity/v3/users/5ccbee1febf64fe394089aa40f58380f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:54:09.755849 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b756205-cb8a-4a3c-abbc-bbe9643e2534 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:09.756176 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b756205-cb8a-4a3c-abbc-bbe9643e2534 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:09.756469 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b756205-cb8a-4a3c-abbc-bbe9643e2534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:09.778189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-5b756205-cb8a-4a3c-abbc-bbe9643e2534 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:54:09.779712 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-5b756205-cb8a-4a3c-abbc-bbe9643e2534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:09.845820 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1467/2909] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:54:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:54:09.858556 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a7bcd-98d0-43d4-a01f-367b434bf60f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:09.860035 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-877a7bcd-98d0-43d4-a01f-367b434bf60f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:09.926112 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:09.947114 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a7bcd-98d0-43d4-a01f-367b434bf60f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:09.951544 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92769b13-4590-4d10-adae-9eba884cdef0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:09.952173 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92769b13-4590-4d10-adae-9eba884cdef0 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:09.952459 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92769b13-4590-4d10-adae-9eba884cdef0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:09.952738 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92769b13-4590-4d10-adae-9eba884cdef0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:09.963586 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92769b13-4590-4d10-adae-9eba884cdef0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:09.965071 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92769b13-4590-4d10-adae-9eba884cdef0 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:09.977297 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1443/2910] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:54:09 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:54:10.291084 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b756205-cb8a-4a3c-abbc-bbe9643e2534 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:10.292736 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-5b756205-cb8a-4a3c-abbc-bbe9643e2534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:10.317069 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b756205-cb8a-4a3c-abbc-bbe9643e2534 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:10.324981 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dec75e-f7c2-4cae-9555-28db6a469872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:10.326118 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dec75e-f7c2-4cae-9555-28db6a469872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:10.326585 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dec75e-f7c2-4cae-9555-28db6a469872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:10.327085 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dec75e-f7c2-4cae-9555-28db6a469872 admin admin] PATH_INFO: `/v3/projects/c5662a2022f44c51bb9e4b989974caf3` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:10.329086 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dec75e-f7c2-4cae-9555-28db6a469872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5662a2022f44c51bb9e4b989974caf3)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:10.333263 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dec75e-f7c2-4cae-9555-28db6a469872 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:10.392388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-18dec75e-f7c2-4cae-9555-28db6a469872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5662a2022f44c51bb9e4b989974caf3', 'request_id': 'req-18dec75e-f7c2-4cae-9555-28db6a469872'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:10.399250 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-18dec75e-f7c2-4cae-9555-28db6a469872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5662a2022f44c51bb9e4b989974caf3', 'request_id': 'req-18dec75e-f7c2-4cae-9555-28db6a469872'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:10.411459 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-18dec75e-f7c2-4cae-9555-28db6a469872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5662a2022f44c51bb9e4b989974caf3', 'request_id': 'req-18dec75e-f7c2-4cae-9555-28db6a469872'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:10.417185 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:10.432837 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-18dec75e-f7c2-4cae-9555-28db6a469872 admin admin] The token cache is being invalidate because project c5662a2022f44c51bb9e4b989974caf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:54:10.433399 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-18dec75e-f7c2-4cae-9555-28db6a469872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:10.434445 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-18dec75e-f7c2-4cae-9555-28db6a469872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:10.436166 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1468/2911] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:54:10 2022] DELETE /identity/v3/projects/c5662a2022f44c51bb9e4b989974caf3 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:54:10.441187 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc86a9-b3fb-4929-8f8d-c5ff5bdd6184 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:10.441731 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc86a9-b3fb-4929-8f8d-c5ff5bdd6184 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:10.441731 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc86a9-b3fb-4929-8f8d-c5ff5bdd6184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:10.464622 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-7bdc86a9-b3fb-4929-8f8d-c5ff5bdd6184 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:54:10.465793 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7bdc86a9-b3fb-4929-8f8d-c5ff5bdd6184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:10.533411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1444/2912] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:54:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:54:10.541602 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915a21d-bbbd-43f0-ad35-4fafd1536e14 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:10.542085 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915a21d-bbbd-43f0-ad35-4fafd1536e14 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:10.542187 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915a21d-bbbd-43f0-ad35-4fafd1536e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:10.563326 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-9915a21d-bbbd-43f0-ad35-4fafd1536e14 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:54:10.564542 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-9915a21d-bbbd-43f0-ad35-4fafd1536e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:10.574553 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1469/2913] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:54:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:54:10.578809 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc86a9-b3fb-4929-8f8d-c5ff5bdd6184 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:10.589764 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc86a9-b3fb-4929-8f8d-c5ff5bdd6184 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:10.594414 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4760f70-1c00-4dc2-a873-a4ba13257dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:10.595025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760f70-1c00-4dc2-a873-a4ba13257dac admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:10.595317 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760f70-1c00-4dc2-a873-a4ba13257dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:10.595590 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760f70-1c00-4dc2-a873-a4ba13257dac admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:10.596282 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4760f70-1c00-4dc2-a873-a4ba13257dac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:54:10.596590 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4760f70-1c00-4dc2-a873-a4ba13257dac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:10.599578 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4760f70-1c00-4dc2-a873-a4ba13257dac admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:10.616648 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1445/2914] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:54:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:54:10.622010 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915a21d-bbbd-43f0-ad35-4fafd1536e14 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:10.632558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915a21d-bbbd-43f0-ad35-4fafd1536e14 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:10.637405 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c406a27-a8da-4e7a-aa7f-55e2f8dcefeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:10.638019 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c406a27-a8da-4e7a-aa7f-55e2f8dcefeb admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:10.638302 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c406a27-a8da-4e7a-aa7f-55e2f8dcefeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:10.638579 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c406a27-a8da-4e7a-aa7f-55e2f8dcefeb admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:10.639258 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c406a27-a8da-4e7a-aa7f-55e2f8dcefeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:10.641203 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c406a27-a8da-4e7a-aa7f-55e2f8dcefeb admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:10.672589 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1470/2915] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:54:10 2022] POST /identity/v3/projects => generated 378 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:54:10.677761 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4760f70-1c00-4dc2-a873-a4ba13257dac admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:10.688565 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4760f70-1c00-4dc2-a873-a4ba13257dac admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:10.756244 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee07b6-a870-48f6-be75-cb6f2c27ce19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:10.757044 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee07b6-a870-48f6-be75-cb6f2c27ce19 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:10.757521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee07b6-a870-48f6-be75-cb6f2c27ce19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:10.757798 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee07b6-a870-48f6-be75-cb6f2c27ce19 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:10.758478 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee07b6-a870-48f6-be75-cb6f2c27ce19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:10.760398 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee07b6-a870-48f6-be75-cb6f2c27ce19 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:10.784598 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1446/2916] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:54:10 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:54:10.789569 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c406a27-a8da-4e7a-aa7f-55e2f8dcefeb admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:10.800280 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c406a27-a8da-4e7a-aa7f-55e2f8dcefeb admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:10.806721 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0971dd0-f254-45b4-a07b-18847023a927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:10.808253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0971dd0-f254-45b4-a07b-18847023a927 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:10.808253 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0971dd0-f254-45b4-a07b-18847023a927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:10.808584 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0971dd0-f254-45b4-a07b-18847023a927 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:10.809382 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0971dd0-f254-45b4-a07b-18847023a927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:10.812193 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0971dd0-f254-45b4-a07b-18847023a927 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:10.834765 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1471/2917] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:54:10 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:54:10.836514 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee07b6-a870-48f6-be75-cb6f2c27ce19 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:10.848145 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee07b6-a870-48f6-be75-cb6f2c27ce19 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:10.856537 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61758a-6180-4219-a1f4-87e36d4a3909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:10.857268 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61758a-6180-4219-a1f4-87e36d4a3909 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:10.857609 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61758a-6180-4219-a1f4-87e36d4a3909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:10.857949 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61758a-6180-4219-a1f4-87e36d4a3909 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:10.858776 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61758a-6180-4219-a1f4-87e36d4a3909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:10.860868 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61758a-6180-4219-a1f4-87e36d4a3909 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:10.875290 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1447/2918] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:54:10 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:54:10.880512 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0971dd0-f254-45b4-a07b-18847023a927 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:10.891335 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0971dd0-f254-45b4-a07b-18847023a927 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:10.896519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee7b2e-6618-4dc8-915b-7085b992b43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:10.896878 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee7b2e-6618-4dc8-915b-7085b992b43e admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:10.897036 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee7b2e-6618-4dc8-915b-7085b992b43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:10.897229 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee7b2e-6618-4dc8-915b-7085b992b43e admin admin] PATH_INFO: `/v3/projects/58301a0e468b43b7bf07f202a1d29d36/users/3f90e5bd1a024a9986b69707ffe088a7/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:10.920007 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee7b2e-6618-4dc8-915b-7085b992b43e admin admin] RBAC: Authorizing `identity:create_grant(project_id=58301a0e468b43b7bf07f202a1d29d36, user_id=3f90e5bd1a024a9986b69707ffe088a7, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:10.922846 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee7b2e-6618-4dc8-915b-7085b992b43e admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:10.938936 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1472/2919] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:54:10 2022] PUT /identity/v3/projects/58301a0e468b43b7bf07f202a1d29d36/users/3f90e5bd1a024a9986b69707ffe088a7/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:54:10.950795 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e7a5b-7aed-491d-8317-5a0b70e2aa29 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:10.951083 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e7a5b-7aed-491d-8317-5a0b70e2aa29 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:10.951357 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e7a5b-7aed-491d-8317-5a0b70e2aa29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:10.981202 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-280e7a5b-7aed-491d-8317-5a0b70e2aa29 None None] MFA Rules not processed for user `3f90e5bd1a024a9986b69707ffe088a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:54:10.982378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-280e7a5b-7aed-491d-8317-5a0b70e2aa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:11.039987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:11.070600 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1448/2920] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:54:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:54:11.076249 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee7b2e-6618-4dc8-915b-7085b992b43e admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:11.146074 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee7b2e-6618-4dc8-915b-7085b992b43e admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:11.150986 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883eadd-68e0-4d17-ae51-7a58b6c8481c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:11.151604 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883eadd-68e0-4d17-ae51-7a58b6c8481c admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:11.151878 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883eadd-68e0-4d17-ae51-7a58b6c8481c admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:11.152154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883eadd-68e0-4d17-ae51-7a58b6c8481c admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:11.152807 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3883eadd-68e0-4d17-ae51-7a58b6c8481c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:11.154739 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3883eadd-68e0-4d17-ae51-7a58b6c8481c admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:11.192009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1473/2921] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:54:11 2022] POST /identity/v3/projects => generated 376 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:54:11.197460 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e7a5b-7aed-491d-8317-5a0b70e2aa29 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:11.208923 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e7a5b-7aed-491d-8317-5a0b70e2aa29 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:11.272156 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4bc477-ef40-400c-b341-e3d74774c88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:11.272791 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bc477-ef40-400c-b341-e3d74774c88c admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:11.273077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bc477-ef40-400c-b341-e3d74774c88c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:11.273394 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bc477-ef40-400c-b341-e3d74774c88c admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:11.274071 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4bc477-ef40-400c-b341-e3d74774c88c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:11.276014 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4bc477-ef40-400c-b341-e3d74774c88c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:11.298533 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1449/2922] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:54:11 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:54:11.303279 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883eadd-68e0-4d17-ae51-7a58b6c8481c admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:11.313694 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883eadd-68e0-4d17-ae51-7a58b6c8481c admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:11.319312 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cedc8-fabe-4101-a609-5a3082f880f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:11.319947 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cedc8-fabe-4101-a609-5a3082f880f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:11.320359 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cedc8-fabe-4101-a609-5a3082f880f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:11.320649 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cedc8-fabe-4101-a609-5a3082f880f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:11.321390 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cedc8-fabe-4101-a609-5a3082f880f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:11.323009 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cedc8-fabe-4101-a609-5a3082f880f0 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:11.338942 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1474/2923] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:54:11 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:54:11.345522 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4bc477-ef40-400c-b341-e3d74774c88c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:11.358585 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4bc477-ef40-400c-b341-e3d74774c88c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:11.363476 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0bc80c-4c4c-48fb-b220-a4fe246bd44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:11.364096 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bc80c-4c4c-48fb-b220-a4fe246bd44c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:11.364911 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bc80c-4c4c-48fb-b220-a4fe246bd44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:11.365335 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bc80c-4c4c-48fb-b220-a4fe246bd44c admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:11.366368 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0bc80c-4c4c-48fb-b220-a4fe246bd44c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:11.367969 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0bc80c-4c4c-48fb-b220-a4fe246bd44c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:11.379198 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1450/2924] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:54:11 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:54:11.385510 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cedc8-fabe-4101-a609-5a3082f880f0 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:11.396899 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cedc8-fabe-4101-a609-5a3082f880f0 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:11.402157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fa774-ba1d-41eb-8744-8fc7fe78ffcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:11.402866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fa774-ba1d-41eb-8744-8fc7fe78ffcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:11.403140 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fa774-ba1d-41eb-8744-8fc7fe78ffcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:11.403416 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fa774-ba1d-41eb-8744-8fc7fe78ffcd admin admin] PATH_INFO: `/v3/projects/30a52c07b74a43cdaec0d29fb3d3a63b/users/cfe4cebf5a7242d99b8c31143d8c980f/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:11.425045 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fa774-ba1d-41eb-8744-8fc7fe78ffcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=30a52c07b74a43cdaec0d29fb3d3a63b, user_id=cfe4cebf5a7242d99b8c31143d8c980f, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:11.428514 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fa774-ba1d-41eb-8744-8fc7fe78ffcd admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:11.444491 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1475/2925] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:54:11 2022] PUT /identity/v3/projects/30a52c07b74a43cdaec0d29fb3d3a63b/users/cfe4cebf5a7242d99b8c31143d8c980f/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:54:11.456789 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a7fa6-8ceb-490a-8af9-144a5cd3104e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:11.457675 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a7fa6-8ceb-490a-8af9-144a5cd3104e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:11.457675 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a7fa6-8ceb-490a-8af9-144a5cd3104e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:11.491680 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-2a5a7fa6-8ceb-490a-8af9-144a5cd3104e None None] MFA Rules not processed for user `cfe4cebf5a7242d99b8c31143d8c980f`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:54:11.492981 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2a5a7fa6-8ceb-490a-8af9-144a5cd3104e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:11.572047 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1451/2926] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:54:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:54:11.577315 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fa774-ba1d-41eb-8744-8fc7fe78ffcd admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:11.631642 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:11.655737 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fa774-ba1d-41eb-8744-8fc7fe78ffcd admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:11.662519 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26012773-1330-47d5-b84a-ec6e5ed7e4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:11.663146 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26012773-1330-47d5-b84a-ec6e5ed7e4c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:11.663436 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26012773-1330-47d5-b84a-ec6e5ed7e4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:11.663710 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26012773-1330-47d5-b84a-ec6e5ed7e4c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:11.664392 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26012773-1330-47d5-b84a-ec6e5ed7e4c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:11.666482 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26012773-1330-47d5-b84a-ec6e5ed7e4c0 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:11.696572 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1476/2927] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:54:11 2022] POST /identity/v3/projects => generated 378 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:54:11.702064 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a7fa6-8ceb-490a-8af9-144a5cd3104e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:11.715480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a7fa6-8ceb-490a-8af9-144a5cd3104e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:11.762399 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:11.777875 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74e727-a326-4bd7-907b-43f4e27d7305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:11.778500 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74e727-a326-4bd7-907b-43f4e27d7305 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:11.778789 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74e727-a326-4bd7-907b-43f4e27d7305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:11.779061 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74e727-a326-4bd7-907b-43f4e27d7305 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:11.779748 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74e727-a326-4bd7-907b-43f4e27d7305 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:11.781695 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74e727-a326-4bd7-907b-43f4e27d7305 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:11.807295 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1452/2928] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:54:11 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:54:11.811637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26012773-1330-47d5-b84a-ec6e5ed7e4c0 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:11.822800 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26012773-1330-47d5-b84a-ec6e5ed7e4c0 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:11.827687 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269774f-7bdb-4da1-b6e3-7fbbae51fb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:11.828400 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269774f-7bdb-4da1-b6e3-7fbbae51fb5a admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:11.828730 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269774f-7bdb-4da1-b6e3-7fbbae51fb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:11.829050 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269774f-7bdb-4da1-b6e3-7fbbae51fb5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:11.829838 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269774f-7bdb-4da1-b6e3-7fbbae51fb5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:11.831502 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269774f-7bdb-4da1-b6e3-7fbbae51fb5a admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:11.843423 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1477/2929] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:54:11 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:54:11.848090 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74e727-a326-4bd7-907b-43f4e27d7305 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:11.859889 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74e727-a326-4bd7-907b-43f4e27d7305 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:11.865514 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0956a75-36d7-4b8f-a0c1-4ac668f74410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:11.866219 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0956a75-36d7-4b8f-a0c1-4ac668f74410 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:11.866499 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0956a75-36d7-4b8f-a0c1-4ac668f74410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:11.866774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0956a75-36d7-4b8f-a0c1-4ac668f74410 admin admin] PATH_INFO: `/v3/projects/ccbc0fe18ad945fbbd63a8e32dc3375d/users/1dfacc6b4baa49d3882d86bc2b26756d/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:11.886900 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0956a75-36d7-4b8f-a0c1-4ac668f74410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccbc0fe18ad945fbbd63a8e32dc3375d, user_id=1dfacc6b4baa49d3882d86bc2b26756d, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:11.892015 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0956a75-36d7-4b8f-a0c1-4ac668f74410 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:11.914000 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1453/2930] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:54:11 2022] PUT /identity/v3/projects/ccbc0fe18ad945fbbd63a8e32dc3375d/users/1dfacc6b4baa49d3882d86bc2b26756d/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:54:11.919010 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269774f-7bdb-4da1-b6e3-7fbbae51fb5a admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:11.932089 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269774f-7bdb-4da1-b6e3-7fbbae51fb5a admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:12.024520 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9318d-6ceb-4b2a-a9fd-3efaa5dfbbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:12.025588 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9318d-6ceb-4b2a-a9fd-3efaa5dfbbd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:12.026064 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9318d-6ceb-4b2a-a9fd-3efaa5dfbbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:12.026532 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9318d-6ceb-4b2a-a9fd-3efaa5dfbbd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:12.027692 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9318d-6ceb-4b2a-a9fd-3efaa5dfbbd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:12.030222 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9318d-6ceb-4b2a-a9fd-3efaa5dfbbd2 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:12.051184 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1478/2931] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:54:11 2022] GET /identity/v3/roles => generated 1427 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:54:12.053701 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0956a75-36d7-4b8f-a0c1-4ac668f74410 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:12.081746 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0956a75-36d7-4b8f-a0c1-4ac668f74410 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:12.092048 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288e6c0-fc8e-4b7c-a06e-50c66450c44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:12.093147 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288e6c0-fc8e-4b7c-a06e-50c66450c44d admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:12.093353 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288e6c0-fc8e-4b7c-a06e-50c66450c44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:12.093510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288e6c0-fc8e-4b7c-a06e-50c66450c44d admin admin] PATH_INFO: `/v3/projects/ccbc0fe18ad945fbbd63a8e32dc3375d/users/1dfacc6b4baa49d3882d86bc2b26756d/roles/7cbff51867a7475d837afd51f6c12f11` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:12.095549 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288e6c0-fc8e-4b7c-a06e-50c66450c44d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccbc0fe18ad945fbbd63a8e32dc3375d, user_id=1dfacc6b4baa49d3882d86bc2b26756d, role_id=7cbff51867a7475d837afd51f6c12f11)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:12.100844 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288e6c0-fc8e-4b7c-a06e-50c66450c44d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:12.117072 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1454/2932] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:54:12 2022] PUT /identity/v3/projects/ccbc0fe18ad945fbbd63a8e32dc3375d/users/1dfacc6b4baa49d3882d86bc2b26756d/roles/7cbff51867a7475d837afd51f6c12f11 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:54:12.130819 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab9fb9-67fd-4cd9-bd5c-04779ced4562 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:12.131277 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab9fb9-67fd-4cd9-bd5c-04779ced4562 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:12.132015 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab9fb9-67fd-4cd9-bd5c-04779ced4562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:12.169783 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-49ab9fb9-67fd-4cd9-bd5c-04779ced4562 None None] MFA Rules not processed for user `1dfacc6b4baa49d3882d86bc2b26756d`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:54:12.171022 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-49ab9fb9-67fd-4cd9-bd5c-04779ced4562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:12.246726 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1479/2933] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:54:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:54:12.261222 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288e6c0-fc8e-4b7c-a06e-50c66450c44d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:12.263025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7288e6c0-fc8e-4b7c-a06e-50c66450c44d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:12.347200 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288e6c0-fc8e-4b7c-a06e-50c66450c44d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:12.353587 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d761d97-e9c1-48ec-ae14-560c602f60e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:12.354469 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d761d97-e9c1-48ec-ae14-560c602f60e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:12.354752 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d761d97-e9c1-48ec-ae14-560c602f60e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:12.355028 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d761d97-e9c1-48ec-ae14-560c602f60e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:12.419045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d761d97-e9c1-48ec-ae14-560c602f60e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:12.420510 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d761d97-e9c1-48ec-ae14-560c602f60e8 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:12.434025 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1455/2934] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:54:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:54:14.019500 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab9fb9-67fd-4cd9-bd5c-04779ced4562 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:14.021698 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-49ab9fb9-67fd-4cd9-bd5c-04779ced4562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:14.034227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab9fb9-67fd-4cd9-bd5c-04779ced4562 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:14.038342 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb149b1-31d3-4a70-9cae-216c6f245fc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:14.039057 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb149b1-31d3-4a70-9cae-216c6f245fc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:14.039343 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb149b1-31d3-4a70-9cae-216c6f245fc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:14.039621 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb149b1-31d3-4a70-9cae-216c6f245fc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:14.075573 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:14.115435 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb149b1-31d3-4a70-9cae-216c6f245fc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:14.117004 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb149b1-31d3-4a70-9cae-216c6f245fc6 service cinder] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:14.126922 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1480/2935] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:54:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:54:14.809047 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d761d97-e9c1-48ec-ae14-560c602f60e8 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:14.819031 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d761d97-e9c1-48ec-ae14-560c602f60e8 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:14.823582 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffde95d-548b-447d-8c48-da9ef9394be7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:14.825267 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffde95d-548b-447d-8c48-da9ef9394be7 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:14.825267 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffde95d-548b-447d-8c48-da9ef9394be7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:14.825267 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffde95d-548b-447d-8c48-da9ef9394be7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:14.835228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffde95d-548b-447d-8c48-da9ef9394be7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:14.836752 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffde95d-548b-447d-8c48-da9ef9394be7 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:14.846546 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1456/2936] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:54:14 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:54:18.901857 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb149b1-31d3-4a70-9cae-216c6f245fc6 service cinder] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:18.974354 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb149b1-31d3-4a70-9cae-216c6f245fc6 service cinder] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:18.980027 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e1293-3e53-4a69-b794-3a8f7c78eaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:18.980727 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e1293-3e53-4a69-b794-3a8f7c78eaa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:18.981059 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e1293-3e53-4a69-b794-3a8f7c78eaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:18.981404 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e1293-3e53-4a69-b794-3a8f7c78eaa5 admin admin] PATH_INFO: `/v3/users/3f90e5bd1a024a9986b69707ffe088a7` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:18.982886 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e1293-3e53-4a69-b794-3a8f7c78eaa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f90e5bd1a024a9986b69707ffe088a7)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:18.985388 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e1293-3e53-4a69-b794-3a8f7c78eaa5 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:19.046768 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-004e1293-3e53-4a69-b794-3a8f7c78eaa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f90e5bd1a024a9986b69707ffe088a7'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:19.047578 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-004e1293-3e53-4a69-b794-3a8f7c78eaa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f90e5bd1a024a9986b69707ffe088a7'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:19.056767 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-004e1293-3e53-4a69-b794-3a8f7c78eaa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f90e5bd1a024a9986b69707ffe088a7'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:19.071055 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-004e1293-3e53-4a69-b794-3a8f7c78eaa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f90e5bd1a024a9986b69707ffe088a7'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:19.078498 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-004e1293-3e53-4a69-b794-3a8f7c78eaa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f90e5bd1a024a9986b69707ffe088a7'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:19.085581 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1481/2937] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:54:18 2022] DELETE /identity/v3/users/3f90e5bd1a024a9986b69707ffe088a7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:54:19.091378 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ce79e-be2b-43b4-9d86-992b46c86616 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:19.091880 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ce79e-be2b-43b4-9d86-992b46c86616 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:19.092356 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ce79e-be2b-43b4-9d86-992b46c86616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:19.115452 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-7d0ce79e-be2b-43b4-9d86-992b46c86616 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:54:19.116684 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7d0ce79e-be2b-43b4-9d86-992b46c86616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:19.168076 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:19.193195 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1457/2938] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:54:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:54:19.204971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e1293-3e53-4a69-b794-3a8f7c78eaa5 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:19.206471 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-004e1293-3e53-4a69-b794-3a8f7c78eaa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:19.281486 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e1293-3e53-4a69-b794-3a8f7c78eaa5 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:19.286608 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce26488-d53c-4bd6-a307-f2ffcecadb32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:19.287230 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce26488-d53c-4bd6-a307-f2ffcecadb32 service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:19.287507 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce26488-d53c-4bd6-a307-f2ffcecadb32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:19.287781 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce26488-d53c-4bd6-a307-f2ffcecadb32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:19.294087 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:19.302154 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce26488-d53c-4bd6-a307-f2ffcecadb32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:19.306199 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce26488-d53c-4bd6-a307-f2ffcecadb32 service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:19.318693 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1482/2939] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:54:19 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:54:19.653414 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ce79e-be2b-43b4-9d86-992b46c86616 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:19.655524 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7d0ce79e-be2b-43b4-9d86-992b46c86616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:19.679986 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ce79e-be2b-43b4-9d86-992b46c86616 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:19.690748 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370ba97-f81c-4b26-bfbc-c7bb0144c535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:19.691852 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370ba97-f81c-4b26-bfbc-c7bb0144c535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:19.693580 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370ba97-f81c-4b26-bfbc-c7bb0144c535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:19.693580 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370ba97-f81c-4b26-bfbc-c7bb0144c535 admin admin] PATH_INFO: `/v3/projects/58301a0e468b43b7bf07f202a1d29d36` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:19.696020 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b370ba97-f81c-4b26-bfbc-c7bb0144c535 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58301a0e468b43b7bf07f202a1d29d36)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:19.698007 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b370ba97-f81c-4b26-bfbc-c7bb0144c535 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:19.771480 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b370ba97-f81c-4b26-bfbc-c7bb0144c535 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58301a0e468b43b7bf07f202a1d29d36', 'request_id': 'req-b370ba97-f81c-4b26-bfbc-c7bb0144c535'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:19.778896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b370ba97-f81c-4b26-bfbc-c7bb0144c535 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58301a0e468b43b7bf07f202a1d29d36', 'request_id': 'req-b370ba97-f81c-4b26-bfbc-c7bb0144c535'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:19.788560 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b370ba97-f81c-4b26-bfbc-c7bb0144c535 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58301a0e468b43b7bf07f202a1d29d36', 'request_id': 'req-b370ba97-f81c-4b26-bfbc-c7bb0144c535'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:19.801292 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b370ba97-f81c-4b26-bfbc-c7bb0144c535 admin admin] The token cache is being invalidate because project 58301a0e468b43b7bf07f202a1d29d36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:54:19.801715 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b370ba97-f81c-4b26-bfbc-c7bb0144c535 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:19.802678 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b370ba97-f81c-4b26-bfbc-c7bb0144c535 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:19.804221 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1458/2940] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:54:19 2022] DELETE /identity/v3/projects/58301a0e468b43b7bf07f202a1d29d36 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:54:19.808881 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce26488-d53c-4bd6-a307-f2ffcecadb32 service neutron] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:19.821531 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce26488-d53c-4bd6-a307-f2ffcecadb32 service neutron] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:19.890967 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3023a-e3b5-4196-bbef-cd82cafaea2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:19.891619 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3023a-e3b5-4196-bbef-cd82cafaea2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:19.891901 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3023a-e3b5-4196-bbef-cd82cafaea2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:19.892234 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3023a-e3b5-4196-bbef-cd82cafaea2f admin admin] PATH_INFO: `/v3/users/cfe4cebf5a7242d99b8c31143d8c980f` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:19.893398 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3023a-e3b5-4196-bbef-cd82cafaea2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfe4cebf5a7242d99b8c31143d8c980f)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:19.895946 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3023a-e3b5-4196-bbef-cd82cafaea2f admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:19.960855 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-27f3023a-e3b5-4196-bbef-cd82cafaea2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfe4cebf5a7242d99b8c31143d8c980f'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:19.961570 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-27f3023a-e3b5-4196-bbef-cd82cafaea2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfe4cebf5a7242d99b8c31143d8c980f'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:19.970269 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-27f3023a-e3b5-4196-bbef-cd82cafaea2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfe4cebf5a7242d99b8c31143d8c980f'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:19.983830 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-27f3023a-e3b5-4196-bbef-cd82cafaea2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfe4cebf5a7242d99b8c31143d8c980f'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:19.999674 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-27f3023a-e3b5-4196-bbef-cd82cafaea2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfe4cebf5a7242d99b8c31143d8c980f'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:20.009913 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1483/2941] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:54:19 2022] DELETE /identity/v3/users/cfe4cebf5a7242d99b8c31143d8c980f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:54:20.335261 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370ba97-f81c-4b26-bfbc-c7bb0144c535 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:20.336923 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b370ba97-f81c-4b26-bfbc-c7bb0144c535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:20.354944 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370ba97-f81c-4b26-bfbc-c7bb0144c535 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:20.471009 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed8ac6-914c-48ad-9ad6-3c04dc1548aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:20.471784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed8ac6-914c-48ad-9ad6-3c04dc1548aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:20.472175 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed8ac6-914c-48ad-9ad6-3c04dc1548aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:20.472559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed8ac6-914c-48ad-9ad6-3c04dc1548aa admin admin] PATH_INFO: `/v3/projects/30a52c07b74a43cdaec0d29fb3d3a63b` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:20.475559 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed8ac6-914c-48ad-9ad6-3c04dc1548aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=30a52c07b74a43cdaec0d29fb3d3a63b)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:20.477774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed8ac6-914c-48ad-9ad6-3c04dc1548aa admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:20.549396 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-34ed8ac6-914c-48ad-9ad6-3c04dc1548aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30a52c07b74a43cdaec0d29fb3d3a63b', 'request_id': 'req-34ed8ac6-914c-48ad-9ad6-3c04dc1548aa'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:20.559723 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-34ed8ac6-914c-48ad-9ad6-3c04dc1548aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30a52c07b74a43cdaec0d29fb3d3a63b', 'request_id': 'req-34ed8ac6-914c-48ad-9ad6-3c04dc1548aa'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:20.571454 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-34ed8ac6-914c-48ad-9ad6-3c04dc1548aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30a52c07b74a43cdaec0d29fb3d3a63b', 'request_id': 'req-34ed8ac6-914c-48ad-9ad6-3c04dc1548aa'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:20.577873 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:20.587390 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-34ed8ac6-914c-48ad-9ad6-3c04dc1548aa admin admin] The token cache is being invalidate because project 30a52c07b74a43cdaec0d29fb3d3a63b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:54:20.587986 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-34ed8ac6-914c-48ad-9ad6-3c04dc1548aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:20.589493 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-34ed8ac6-914c-48ad-9ad6-3c04dc1548aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:20.591253 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1459/2942] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:54:20 2022] DELETE /identity/v3/projects/30a52c07b74a43cdaec0d29fb3d3a63b => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:54:20.595661 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3023a-e3b5-4196-bbef-cd82cafaea2f admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:20.597659 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-27f3023a-e3b5-4196-bbef-cd82cafaea2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:20.612971 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3023a-e3b5-4196-bbef-cd82cafaea2f admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:20.691527 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987738fb-98f9-4695-8271-ff30d00b77cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:20.692572 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987738fb-98f9-4695-8271-ff30d00b77cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:20.692860 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987738fb-98f9-4695-8271-ff30d00b77cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:20.693161 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987738fb-98f9-4695-8271-ff30d00b77cd admin admin] PATH_INFO: `/v3/users/1dfacc6b4baa49d3882d86bc2b26756d` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:20.694500 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987738fb-98f9-4695-8271-ff30d00b77cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dfacc6b4baa49d3882d86bc2b26756d)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:20.696506 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987738fb-98f9-4695-8271-ff30d00b77cd admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:20.711773 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:20.748366 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-987738fb-98f9-4695-8271-ff30d00b77cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dfacc6b4baa49d3882d86bc2b26756d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:20.748927 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-987738fb-98f9-4695-8271-ff30d00b77cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dfacc6b4baa49d3882d86bc2b26756d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:20.763412 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-987738fb-98f9-4695-8271-ff30d00b77cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dfacc6b4baa49d3882d86bc2b26756d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:20.777445 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-987738fb-98f9-4695-8271-ff30d00b77cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dfacc6b4baa49d3882d86bc2b26756d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:20.784783 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-987738fb-98f9-4695-8271-ff30d00b77cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dfacc6b4baa49d3882d86bc2b26756d'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:20.792230 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1484/2943] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:54:20 2022] DELETE /identity/v3/users/1dfacc6b4baa49d3882d86bc2b26756d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:54:21.107304 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed8ac6-914c-48ad-9ad6-3c04dc1548aa admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:21.108777 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-34ed8ac6-914c-48ad-9ad6-3c04dc1548aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:21.122155 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed8ac6-914c-48ad-9ad6-3c04dc1548aa admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:21.195854 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb50cb-4a72-4c61-a855-1a5ee6869f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:21.196516 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb50cb-4a72-4c61-a855-1a5ee6869f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:21.196837 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb50cb-4a72-4c61-a855-1a5ee6869f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:21.197136 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb50cb-4a72-4c61-a855-1a5ee6869f22 admin admin] PATH_INFO: `/v3/projects/ccbc0fe18ad945fbbd63a8e32dc3375d` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:21.198348 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb50cb-4a72-4c61-a855-1a5ee6869f22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccbc0fe18ad945fbbd63a8e32dc3375d)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:21.200267 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb50cb-4a72-4c61-a855-1a5ee6869f22 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:21.269217 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-79bb50cb-4a72-4c61-a855-1a5ee6869f22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccbc0fe18ad945fbbd63a8e32dc3375d', 'request_id': 'req-79bb50cb-4a72-4c61-a855-1a5ee6869f22'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:21.277621 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-79bb50cb-4a72-4c61-a855-1a5ee6869f22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccbc0fe18ad945fbbd63a8e32dc3375d', 'request_id': 'req-79bb50cb-4a72-4c61-a855-1a5ee6869f22'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:21.294231 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-79bb50cb-4a72-4c61-a855-1a5ee6869f22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccbc0fe18ad945fbbd63a8e32dc3375d', 'request_id': 'req-79bb50cb-4a72-4c61-a855-1a5ee6869f22'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:21.307220 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-79bb50cb-4a72-4c61-a855-1a5ee6869f22 admin admin] The token cache is being invalidate because project ccbc0fe18ad945fbbd63a8e32dc3375d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:54:21.307517 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-79bb50cb-4a72-4c61-a855-1a5ee6869f22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:21.308204 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-79bb50cb-4a72-4c61-a855-1a5ee6869f22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:21.309930 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1460/2944] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:54:21 2022] DELETE /identity/v3/projects/ccbc0fe18ad945fbbd63a8e32dc3375d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:54:21.315333 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a7d7a-6b17-4745-a5f8-c4c3ee0c1ec1 None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:21.315876 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a7d7a-6b17-4745-a5f8-c4c3ee0c1ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:21.315981 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a7d7a-6b17-4745-a5f8-c4c3ee0c1ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:21.339583 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-337a7d7a-6b17-4745-a5f8-c4c3ee0c1ec1 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:54:21.340993 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-337a7d7a-6b17-4745-a5f8-c4c3ee0c1ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:21.407149 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1485/2945] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:54:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:54:21.416742 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec5676-b19e-448b-8db1-b18e56d1f067 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:21.417339 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec5676-b19e-448b-8db1-b18e56d1f067 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:21.417339 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec5676-b19e-448b-8db1-b18e56d1f067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:21.438563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-20ec5676-b19e-448b-8db1-b18e56d1f067 None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:54:21.439784 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-20ec5676-b19e-448b-8db1-b18e56d1f067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:21.448737 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1461/2946] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:54:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:54:21.454454 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a7d7a-6b17-4745-a5f8-c4c3ee0c1ec1 None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:21.466150 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a7d7a-6b17-4745-a5f8-c4c3ee0c1ec1 None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:21.475964 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc7206-e22a-4053-883b-39e7f192076f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:21.476985 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc7206-e22a-4053-883b-39e7f192076f admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:21.477493 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc7206-e22a-4053-883b-39e7f192076f admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:21.477968 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc7206-e22a-4053-883b-39e7f192076f admin admin] PATH_INFO: `/v3/domains` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:21.479343 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc7206-e22a-4053-883b-39e7f192076f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:54:21.479892 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc7206-e22a-4053-883b-39e7f192076f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:21.483020 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc7206-e22a-4053-883b-39e7f192076f admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:21.498738 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1486/2947] 10.222.0.100 () {58 vars in 1158 bytes} [Tue Aug 2 09:54:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:54:21.506370 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec5676-b19e-448b-8db1-b18e56d1f067 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:21.523490 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec5676-b19e-448b-8db1-b18e56d1f067 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:21.528878 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4be4b-1c6a-4443-b8bb-9a59798a7d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:21.529549 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4be4b-1c6a-4443-b8bb-9a59798a7d74 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:21.529831 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4be4b-1c6a-4443-b8bb-9a59798a7d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:21.530121 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4be4b-1c6a-4443-b8bb-9a59798a7d74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:21.530811 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4be4b-1c6a-4443-b8bb-9a59798a7d74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:21.532729 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4be4b-1c6a-4443-b8bb-9a59798a7d74 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:21.564774 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1462/2948] 10.222.0.100 () {60 vars in 1158 bytes} [Tue Aug 2 09:54:21 2022] POST /identity/v3/projects => generated 390 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:54:21.570013 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc7206-e22a-4053-883b-39e7f192076f admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:21.580944 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc7206-e22a-4053-883b-39e7f192076f admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:21.643864 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50211eeb-79d7-4e0c-bab4-6445eec35ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:21.644488 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50211eeb-79d7-4e0c-bab4-6445eec35ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:21.644790 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50211eeb-79d7-4e0c-bab4-6445eec35ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:21.645433 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50211eeb-79d7-4e0c-bab4-6445eec35ca7 admin admin] PATH_INFO: `/v3/users` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:21.645723 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50211eeb-79d7-4e0c-bab4-6445eec35ca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:21.647920 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50211eeb-79d7-4e0c-bab4-6445eec35ca7 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:21.673227 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1487/2949] 10.222.0.100 () {60 vars in 1149 bytes} [Tue Aug 2 09:54:21 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:54:21.678003 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4be4b-1c6a-4443-b8bb-9a59798a7d74 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:21.694548 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4be4b-1c6a-4443-b8bb-9a59798a7d74 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:21.699411 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0872dc-7335-41a9-960d-1b2b954282dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:21.700042 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0872dc-7335-41a9-960d-1b2b954282dd admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:21.700328 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0872dc-7335-41a9-960d-1b2b954282dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:21.700607 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0872dc-7335-41a9-960d-1b2b954282dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:21.701340 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0872dc-7335-41a9-960d-1b2b954282dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:21.703883 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0872dc-7335-41a9-960d-1b2b954282dd admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:21.716216 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1463/2950] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:54:21 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:54:21.720954 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50211eeb-79d7-4e0c-bab4-6445eec35ca7 admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:21.732360 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50211eeb-79d7-4e0c-bab4-6445eec35ca7 admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:21.737419 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e3cdc-3789-455b-acf1-652a773772b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:21.738044 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3cdc-3789-455b-acf1-652a773772b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:21.738325 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3cdc-3789-455b-acf1-652a773772b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:21.738601 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3cdc-3789-455b-acf1-652a773772b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:21.740293 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e3cdc-3789-455b-acf1-652a773772b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:21.741802 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e3cdc-3789-455b-acf1-652a773772b4 admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:21.744344 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:21.757960 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1488/2951] 10.222.0.100 () {58 vars in 1127 bytes} [Tue Aug 2 09:54:21 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:54:21.763021 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0872dc-7335-41a9-960d-1b2b954282dd admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:21.774482 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0872dc-7335-41a9-960d-1b2b954282dd admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:21.779966 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebb12b-da46-4ed8-81ab-9419af3e5985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:21.780859 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebb12b-da46-4ed8-81ab-9419af3e5985 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:21.781292 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebb12b-da46-4ed8-81ab-9419af3e5985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:21.781460 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebb12b-da46-4ed8-81ab-9419af3e5985 admin admin] PATH_INFO: `/v3/projects/00e64cf880084e5fb0d4e40877f77943/users/5bd209a16ad8494499f950161afdf907/roles/f739a9c857cd4050896a6814c8bc0120` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:21.799639 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adebb12b-da46-4ed8-81ab-9419af3e5985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00e64cf880084e5fb0d4e40877f77943, user_id=5bd209a16ad8494499f950161afdf907, role_id=f739a9c857cd4050896a6814c8bc0120)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:21.803126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adebb12b-da46-4ed8-81ab-9419af3e5985 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:21.819236 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1464/2952] 10.222.0.100 () {60 vars in 1488 bytes} [Tue Aug 2 09:54:21 2022] PUT /identity/v3/projects/00e64cf880084e5fb0d4e40877f77943/users/5bd209a16ad8494499f950161afdf907/roles/f739a9c857cd4050896a6814c8bc0120 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:54:21.833157 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1cf8a-639b-400d-b1ec-e852caedfc0d None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:21.833465 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1cf8a-639b-400d-b1ec-e852caedfc0d None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:21.833747 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1cf8a-639b-400d-b1ec-e852caedfc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:21.862655 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-e2f1cf8a-639b-400d-b1ec-e852caedfc0d None None] MFA Rules not processed for user `5bd209a16ad8494499f950161afdf907`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:54:21.864824 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-e2f1cf8a-639b-400d-b1ec-e852caedfc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:21.938314 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1489/2953] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:54:21 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:54:21.949635 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebb12b-da46-4ed8-81ab-9419af3e5985 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:21.951354 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-adebb12b-da46-4ed8-81ab-9419af3e5985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:21.983228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:22.021245 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebb12b-da46-4ed8-81ab-9419af3e5985 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:22.025182 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795c87d-6340-4da0-8b4f-c999dec20506 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e55bb6b624c45a3a08306403570177d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:22.025737 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795c87d-6340-4da0-8b4f-c999dec20506 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:22.026027 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795c87d-6340-4da0-8b4f-c999dec20506 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:22.026327 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795c87d-6340-4da0-8b4f-c999dec20506 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:22.035333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a795c87d-6340-4da0-8b4f-c999dec20506 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:22.037953 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a795c87d-6340-4da0-8b4f-c999dec20506 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:22.047733 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1465/2954] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:54:21 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:54:31.935035 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=33 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:32.043084 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=16 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:41.603210 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1cf8a-639b-400d-b1ec-e852caedfc0d None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:41.676458 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1cf8a-639b-400d-b1ec-e852caedfc0d None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:41.681189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153c153-26bd-456b-b1fb-58bda483b90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:41.681833 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153c153-26bd-456b-b1fb-58bda483b90c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:41.682115 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153c153-26bd-456b-b1fb-58bda483b90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:41.682391 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153c153-26bd-456b-b1fb-58bda483b90c admin admin] PATH_INFO: `/v3/users/5bd209a16ad8494499f950161afdf907` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:41.683558 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153c153-26bd-456b-b1fb-58bda483b90c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bd209a16ad8494499f950161afdf907)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:41.685527 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153c153-26bd-456b-b1fb-58bda483b90c admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:41.735129 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-f153c153-26bd-456b-b1fb-58bda483b90c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bd209a16ad8494499f950161afdf907'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:41.735803 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-f153c153-26bd-456b-b1fb-58bda483b90c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bd209a16ad8494499f950161afdf907'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:41.753036 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-f153c153-26bd-456b-b1fb-58bda483b90c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bd209a16ad8494499f950161afdf907'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:41.770906 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-f153c153-26bd-456b-b1fb-58bda483b90c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bd209a16ad8494499f950161afdf907'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:41.778900 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-f153c153-26bd-456b-b1fb-58bda483b90c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bd209a16ad8494499f950161afdf907'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:41.785930 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1490/2955] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:54:41 2022] DELETE /identity/v3/users/5bd209a16ad8494499f950161afdf907 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:54:41.790595 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7547432-1155-4d07-b8ba-868551b66a9f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:41.791360 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7547432-1155-4d07-b8ba-868551b66a9f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:41.791360 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7547432-1155-4d07-b8ba-868551b66a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:41.810921 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a7547432-1155-4d07-b8ba-868551b66a9f None None] MFA Rules not processed for user `87e4d594c1134b86837b4e4c464d9cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:54:41.812126 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a7547432-1155-4d07-b8ba-868551b66a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:41.878359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1466/2956] 10.222.0.100 () {58 vars in 963 bytes} [Tue Aug 2 09:54:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:54:41.888112 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153c153-26bd-456b-b1fb-58bda483b90c admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:41.889580 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-f153c153-26bd-456b-b1fb-58bda483b90c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:41.962158 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153c153-26bd-456b-b1fb-58bda483b90c admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:41.965984 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c6eff-6d12-43e0-bc18-465c8a62da9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4afef33deb247509765a8c88a78ac39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:41.966613 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c6eff-6d12-43e0-bc18-465c8a62da9a service neutron] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:41.966900 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c6eff-6d12-43e0-bc18-465c8a62da9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:41.967178 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c6eff-6d12-43e0-bc18-465c8a62da9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:41.977575 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4c6eff-6d12-43e0-bc18-465c8a62da9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:41.979040 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4c6eff-6d12-43e0-bc18-465c8a62da9a service neutron] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:41.989189 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1491/2957] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:54:41 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:54:42.278044 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7547432-1155-4d07-b8ba-868551b66a9f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:42.279646 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a7547432-1155-4d07-b8ba-868551b66a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:42.293515 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7547432-1155-4d07-b8ba-868551b66a9f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:42.298837 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af2d0d-f27f-41d1-bb67-db0132497347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:42.299569 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af2d0d-f27f-41d1-bb67-db0132497347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:42.299968 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af2d0d-f27f-41d1-bb67-db0132497347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:42.300326 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af2d0d-f27f-41d1-bb67-db0132497347 admin admin] PATH_INFO: `/v3/projects/00e64cf880084e5fb0d4e40877f77943` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:42.301889 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af2d0d-f27f-41d1-bb67-db0132497347 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00e64cf880084e5fb0d4e40877f77943)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:42.303834 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af2d0d-f27f-41d1-bb67-db0132497347 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:42.354847 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c6af2d0d-f27f-41d1-bb67-db0132497347 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00e64cf880084e5fb0d4e40877f77943', 'request_id': 'req-c6af2d0d-f27f-41d1-bb67-db0132497347'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:42.361228 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c6af2d0d-f27f-41d1-bb67-db0132497347 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00e64cf880084e5fb0d4e40877f77943', 'request_id': 'req-c6af2d0d-f27f-41d1-bb67-db0132497347'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:42.369834 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c6af2d0d-f27f-41d1-bb67-db0132497347 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00e64cf880084e5fb0d4e40877f77943', 'request_id': 'req-c6af2d0d-f27f-41d1-bb67-db0132497347'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:42.384764 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c6af2d0d-f27f-41d1-bb67-db0132497347 admin admin] The token cache is being invalidate because project 00e64cf880084e5fb0d4e40877f77943 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:54:42.385001 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c6af2d0d-f27f-41d1-bb67-db0132497347 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:42.385687 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c6af2d0d-f27f-41d1-bb67-db0132497347 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:54:42.387290 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1467/2958] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:54:42 2022] DELETE /identity/v3/projects/00e64cf880084e5fb0d4e40877f77943 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:54:51.984468 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:52.381359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:55:00.018273 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bde69-b680-4dfa-8a2d-9fc5bb01d44b None None] REQUEST_METHOD: `POST` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:00.019308 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bde69-b680-4dfa-8a2d-9fc5bb01d44b None None] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:00.019481 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bde69-b680-4dfa-8a2d-9fc5bb01d44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:00.039397 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.auth.core [None req-561bde69-b680-4dfa-8a2d-9fc5bb01d44b None None] MFA Rules not processed for user `1cd36b543bd9487ebd8ce578e9a76c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:55:00.040534 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-561bde69-b680-4dfa-8a2d-9fc5bb01d44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:00.099202 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1492/2959] 10.222.0.100 () {58 vars in 972 bytes} [Tue Aug 2 09:55:00 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:55:00.141166 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af2d0d-f27f-41d1-bb67-db0132497347 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:55:00.142600 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c6af2d0d-f27f-41d1-bb67-db0132497347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:00.214796 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af2d0d-f27f-41d1-bb67-db0132497347 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:55:00.219914 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c7e46-9490-4926-a1b4-fb8b03a790e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50cb532b58ed4951ba960a0a7d579b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:55:00.220545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c7e46-9490-4926-a1b4-fb8b03a790e2 service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:00.220835 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c7e46-9490-4926-a1b4-fb8b03a790e2 service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:00.221132 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c7e46-9490-4926-a1b4-fb8b03a790e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:00.230074 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c7e46-9490-4926-a1b4-fb8b03a790e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:55:00.231669 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c7e46-9490-4926-a1b4-fb8b03a790e2 service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:55:00.240571 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1468/2960] 10.222.0.100 () {60 vars in 1389 bytes} [Tue Aug 2 09:55:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 02 09:55:03.638633 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bde69-b680-4dfa-8a2d-9fc5bb01d44b None None] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:55:03.640554 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-561bde69-b680-4dfa-8a2d-9fc5bb01d44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:03.734427 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bde69-b680-4dfa-8a2d-9fc5bb01d44b None None] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:55:03.743074 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b22ee-20eb-4464-a1d7-54599f6a9eab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b40981fe179a43dc81553d3594dad649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:55:03.744162 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b22ee-20eb-4464-a1d7-54599f6a9eab service placement] REQUEST_METHOD: `GET` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:03.744636 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b22ee-20eb-4464-a1d7-54599f6a9eab service placement] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:03.745274 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b22ee-20eb-4464-a1d7-54599f6a9eab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:03.747777 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-066b22ee-20eb-4464-a1d7-54599f6a9eab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:03.766105 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b22ee-20eb-4464-a1d7-54599f6a9eab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:55:03.768238 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b22ee-20eb-4464-a1d7-54599f6a9eab service placement] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:55:03.785320 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1493/2961] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:55:03 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:55:07.064566 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c7e46-9490-4926-a1b4-fb8b03a790e2 service swift] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:55:07.066960 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-be5c7e46-9490-4926-a1b4-fb8b03a790e2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:07.151218 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c7e46-9490-4926-a1b4-fb8b03a790e2 service swift] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:55:07.155853 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058465ff-0465-4593-abbb-06d700dc44f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:55:07.156505 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058465ff-0465-4593-abbb-06d700dc44f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:07.156800 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058465ff-0465-4593-abbb-06d700dc44f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:07.157077 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058465ff-0465-4593-abbb-06d700dc44f6 admin admin] PATH_INFO: `/v3/users/0ea7a204b1d34f7dacfe9f468f565e65` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:07.159181 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058465ff-0465-4593-abbb-06d700dc44f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ea7a204b1d34f7dacfe9f468f565e65)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:55:07.161168 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058465ff-0465-4593-abbb-06d700dc44f6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:55:07.229092 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-058465ff-0465-4593-abbb-06d700dc44f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ea7a204b1d34f7dacfe9f468f565e65'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:07.236987 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-058465ff-0465-4593-abbb-06d700dc44f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ea7a204b1d34f7dacfe9f468f565e65'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:07.239449 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-058465ff-0465-4593-abbb-06d700dc44f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ea7a204b1d34f7dacfe9f468f565e65'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:07.249707 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-058465ff-0465-4593-abbb-06d700dc44f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ea7a204b1d34f7dacfe9f468f565e65'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:07.250502 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-058465ff-0465-4593-abbb-06d700dc44f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ea7a204b1d34f7dacfe9f468f565e65'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:07.283896 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1469/2962] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:55:07 2022] DELETE /identity/v3/users/0ea7a204b1d34f7dacfe9f468f565e65 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:55:07.484602 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b22ee-20eb-4464-a1d7-54599f6a9eab service placement] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:55:07.499330 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b22ee-20eb-4464-a1d7-54599f6a9eab service placement] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:55:07.560397 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943c67a-d6dc-4de5-b463-4d7b7ac79aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:55:07.561041 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943c67a-d6dc-4de5-b463-4d7b7ac79aab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:07.561355 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943c67a-d6dc-4de5-b463-4d7b7ac79aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:07.561638 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943c67a-d6dc-4de5-b463-4d7b7ac79aab admin admin] PATH_INFO: `/v3/projects/557cbe65acb94a749a05118b3f3b70ff` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:07.562787 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943c67a-d6dc-4de5-b463-4d7b7ac79aab admin admin] RBAC: Authorizing `identity:delete_project(project_id=557cbe65acb94a749a05118b3f3b70ff)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:55:07.564998 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943c67a-d6dc-4de5-b463-4d7b7ac79aab admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:55:07.607234 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:55:07.616824 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d943c67a-d6dc-4de5-b463-4d7b7ac79aab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '557cbe65acb94a749a05118b3f3b70ff', 'request_id': 'req-d943c67a-d6dc-4de5-b463-4d7b7ac79aab'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:07.622955 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d943c67a-d6dc-4de5-b463-4d7b7ac79aab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '557cbe65acb94a749a05118b3f3b70ff', 'request_id': 'req-d943c67a-d6dc-4de5-b463-4d7b7ac79aab'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:07.631608 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d943c67a-d6dc-4de5-b463-4d7b7ac79aab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '557cbe65acb94a749a05118b3f3b70ff', 'request_id': 'req-d943c67a-d6dc-4de5-b463-4d7b7ac79aab'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:07.643144 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d943c67a-d6dc-4de5-b463-4d7b7ac79aab admin admin] The token cache is being invalidate because project 557cbe65acb94a749a05118b3f3b70ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:55:07.643516 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d943c67a-d6dc-4de5-b463-4d7b7ac79aab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:07.644701 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-d943c67a-d6dc-4de5-b463-4d7b7ac79aab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:07.646278 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1494/2963] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:55:07 2022] DELETE /identity/v3/projects/557cbe65acb94a749a05118b3f3b70ff => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:55:07.650521 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058465ff-0465-4593-abbb-06d700dc44f6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:55:07.652042 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-058465ff-0465-4593-abbb-06d700dc44f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:07.666150 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058465ff-0465-4593-abbb-06d700dc44f6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:55:07.670954 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:55:07.726708 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef812c-47dd-4059-a522-50d17f7c48cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:55:07.727264 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef812c-47dd-4059-a522-50d17f7c48cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:07.727584 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef812c-47dd-4059-a522-50d17f7c48cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:07.727859 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef812c-47dd-4059-a522-50d17f7c48cf admin admin] PATH_INFO: `/v3/users/273a79f0fa4e4b1eb3f36866396a34a9` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:07.729166 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef812c-47dd-4059-a522-50d17f7c48cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=273a79f0fa4e4b1eb3f36866396a34a9)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:55:07.731111 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef812c-47dd-4059-a522-50d17f7c48cf admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:55:07.781470 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cfef812c-47dd-4059-a522-50d17f7c48cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '273a79f0fa4e4b1eb3f36866396a34a9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:07.789537 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cfef812c-47dd-4059-a522-50d17f7c48cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '273a79f0fa4e4b1eb3f36866396a34a9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:07.790212 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cfef812c-47dd-4059-a522-50d17f7c48cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '273a79f0fa4e4b1eb3f36866396a34a9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:07.797555 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cfef812c-47dd-4059-a522-50d17f7c48cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '273a79f0fa4e4b1eb3f36866396a34a9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:07.798333 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cfef812c-47dd-4059-a522-50d17f7c48cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '273a79f0fa4e4b1eb3f36866396a34a9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:07.816283 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1470/2964] 10.222.0.100 () {58 vars in 1229 bytes} [Tue Aug 2 09:55:07 2022] DELETE /identity/v3/users/273a79f0fa4e4b1eb3f36866396a34a9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:55:07.993866 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943c67a-d6dc-4de5-b463-4d7b7ac79aab admin admin] Authenticating user token {{(pid=80361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:55:07.995321 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.fernet_utils [None req-d943c67a-d6dc-4de5-b463-4d7b7ac79aab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:08.020357 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943c67a-d6dc-4de5-b463-4d7b7ac79aab admin admin] Validating token access rules against request {{(pid=80361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:55:08.082124 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a9da8-3047-4fd9-82ca-aabd09a1cc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e4d594c1134b86837b4e4c464d9cfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b02845ab43a49ada6db1e8f52cf0026', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:55:08.082776 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a9da8-3047-4fd9-82ca-aabd09a1cc1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:08.083061 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a9da8-3047-4fd9-82ca-aabd09a1cc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:08.083342 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a9da8-3047-4fd9-82ca-aabd09a1cc1b admin admin] PATH_INFO: `/v3/projects/da3b6337b1924f6c95d749b0e4ef17d1` {{(pid=80361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:08.084512 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a9da8-3047-4fd9-82ca-aabd09a1cc1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=da3b6337b1924f6c95d749b0e4ef17d1)` {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:55:08.086508 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a9da8-3047-4fd9-82ca-aabd09a1cc1b admin admin] RBAC: Authorization granted {{(pid=80361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:55:08.134637 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-cf3a9da8-3047-4fd9-82ca-aabd09a1cc1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da3b6337b1924f6c95d749b0e4ef17d1', 'request_id': 'req-cf3a9da8-3047-4fd9-82ca-aabd09a1cc1b'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:08.140681 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-cf3a9da8-3047-4fd9-82ca-aabd09a1cc1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da3b6337b1924f6c95d749b0e4ef17d1', 'request_id': 'req-cf3a9da8-3047-4fd9-82ca-aabd09a1cc1b'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:08.147750 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-cf3a9da8-3047-4fd9-82ca-aabd09a1cc1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da3b6337b1924f6c95d749b0e4ef17d1', 'request_id': 'req-cf3a9da8-3047-4fd9-82ca-aabd09a1cc1b'} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:08.158546 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-cf3a9da8-3047-4fd9-82ca-aabd09a1cc1b admin admin] The token cache is being invalidate because project da3b6337b1924f6c95d749b0e4ef17d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:55:08.158835 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-cf3a9da8-3047-4fd9-82ca-aabd09a1cc1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:08.160197 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG keystone.notifications [None req-cf3a9da8-3047-4fd9-82ca-aabd09a1cc1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:08.161723 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: [pid: 80361|app: 0|req: 1495/2965] 10.222.0.100 () {58 vars in 1238 bytes} [Tue Aug 2 09:55:07 2022] DELETE /identity/v3/projects/da3b6337b1924f6c95d749b0e4ef17d1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:55:17.812547 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:55:18.155345 dv-cinder-803823-15-iscsi devstack@keystone.service[80361]: DEBUG dbcounter [-] [80361] Writing DB stats keystone:SELECT=62,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:55:45.118359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef812c-47dd-4059-a522-50d17f7c48cf admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:55:45.120359 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-cfef812c-47dd-4059-a522-50d17f7c48cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:45.198890 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef812c-47dd-4059-a522-50d17f7c48cf admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:55:45.264545 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c55961-1d1c-4042-be00-d38ee28c9a56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b40981fe179a43dc81553d3594dad649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61ca0176324e445b94e42a5214d2c89b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:55:45.265199 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c55961-1d1c-4042-be00-d38ee28c9a56 service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:45.265489 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c55961-1d1c-4042-be00-d38ee28c9a56 service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:45.265772 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c55961-1d1c-4042-be00-d38ee28c9a56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:45.267563 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d3c55961-1d1c-4042-be00-d38ee28c9a56 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:45.280045 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c55961-1d1c-4042-be00-d38ee28c9a56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:55:45.281830 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c55961-1d1c-4042-be00-d38ee28c9a56 service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:55:45.293062 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1471/2966] 10.222.0.100 () {60 vars in 1370 bytes} [Tue Aug 2 09:55:45 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:55:55.288709 dv-cinder-803823-15-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=58 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}